ML.NET and Unity #1886
Comments
|
Hi @mihaar , Thanks for filing this issue! Technically, ML.NET should work with Unity, but we haven't done comprehensive testing for Unity yet. We've added this for the 0.10 schedule (starting early January, releasing at the end of January). One question: What version of Unity are you using? |
|
..I am using unity version: 2018.3.0f2 Personal. |
|
looking forward to see it works with Unity. and announce sooner in the end of coming Jan. Though we are now seeking for the tensorflow complied with Unity. |
|
@rogancarr I have managet to setup unity in a way I can now import ML dlls and train models, but when trying to make a prediction I came across this error: (similar code works in .net framework 4.7.2 solution) Example is attached |
|
Don't know if this helps, but the iris example in unity returns the same error: ArgumentException: method arguments are incompatible. |
|
As sugested by @shauheen I have tryed importing into Unity and training ML version 0.9.0. The cal: (mind you, method MakePredictionFunction is now renamed to CreatePredictionEngine, but it seems the error is the same) |
EDIT: Aivaras Ivoškus from Unity says: "The issue does not seem to be related to Scripting Restrictions of Unity Engine. "@shauheen @rogancarr @danmosemsft UNITY SCRIPTING RESTRICTIONS
I think the method Microsoft.ML.ApiUtils.GeneratePeek[TOwn,TRow] (Microsoft.ML.Data.InternalSchemaDefinition+Column column) (at <22a400311822408a8bd05691f3cdc119>:0) does that exactly, or am I wrong? |
|
On sugestion from Unity QA team I have tryed building the app containing ML apis with IL2CPP scripting backend, but got this errors: IL2CPP error for type 'Microsoft.ML.Transforms.RowShufflingTransformer/Cursor/ShufflePipe/ImplVec Failed running C:\Program Files\Unity\Editor\Data\il2cpp/build/il2cpp.exe --convert-to-cpp --emit-null-checks --enable-array-bounds-check --dotnetprofile="unityaot" --enable-debugger --compile-cpp --libil2cpp-static --platform="WindowsDesktop" --architecture="x64" --configuration="Release" --outputpath="C:\Users\uporabnik\Desktop\newTestCanMlaiDELETE\Temp/StagingArea/Data\Native\GameAssembly.dll" --cachedirectory="C:\Users\uporabnik\Desktop\newTestCanMlaiDELETE\Assets..\Library/il2cpp_cache" --map-file-parser="C:\Program Files\Unity\Editor\Data\Tools\MapFileParser\MapFileParser.exe" --directory="C:\Users\uporabnik\Desktop\newTestCanMlaiDELETE\Temp\StagingArea\Data\Managed" --generatedcppdir="C:\Users\uporabnik\Desktop\newTestCanMlaiDELETE\Temp\StagingArea\Data\il2cppOutput" stdout: Unhandled Exception: System.Exception: Invalid method 'null' found in vtable for 'Microsoft.ML.Transforms.RowShufflingTransformer/Cursor/ShufflePipe/ImplVec UnityEngine.Debug:LogError(Object) Can build with mono scripting backend though. |
|
Myself and @eerhardt spent some time with Unity 2018.3.4f1 version and have got a basic ML.NET model scoring working. These are primary workarounds we made.
Here's a unity package which shows this play. Does this help @mihaar? |
|
Your example works also with binaries (0.9.0) I imported into my project.
But I do not know how to implement it into a different project.
Can you please do the iris example or the taxi fare example:
https://docs.microsoft.com/en-us/dotnet/machine-learning/tutorials/taxi-fare
Thanks !
…On Wed, Feb 6, 2019 at 8:25 AM Ankit Asthana ***@***.***> wrote:
Myself and @eerhardt <https://github.com/eerhardt> spent some time with
Unity 2018.3.4f1 version and have got a basic ML.NET model scoring
working. These are primary workarounds we made.
- Create a plugins folder in assets and add core ML.NET Nuget binaries
along with all nested
dependencies (e.g. Sytem.Memory) and inherent native dependencies
(e.g. CPUMathNative)
- Use lower-level API to score ML.NET models and avoid the high level
convinience APIs e.g.
PredictionEngine as they currently make use of reflection emit which
throws up.
- Target .NET 4.x API compatibility level (default for 2018.3.4f1)
Here's a unity package which shows this play.
https://github.com/dotnet/machinelearning-samples/tree/master/samples/csharp/end-to-end-apps/Unity-HelloMLNET
Does this help @mihaar <https://github.com/mihaar>?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#1886 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/Aj1kTAya0FRTJzsOj1u9P15IKc6Vwkf_ks5vKoNhgaJpZM4ZU87E>
.
|
|
ps It is not wisible in your example how to train the model to get it in
the right 'format' so that it wil take floats as parameters
On Thu, Feb 7, 2019 at 12:06 PM đibril fashrista <[email protected]>
wrote:
… Your example works also with binaries (0.9.0) I imported into my project.
But I do not know how to implement it into a different project.
Can you please do the iris example or the taxi fare example:
https://docs.microsoft.com/en-us/dotnet/machine-learning/tutorials/taxi-fare
Thanks !
On Wed, Feb 6, 2019 at 8:25 AM Ankit Asthana ***@***.***>
wrote:
> Myself and @eerhardt <https://github.com/eerhardt> spent some time with
> Unity 2018.3.4f1 version and have got a basic ML.NET model scoring
> working. These are primary workarounds we made.
>
> - Create a plugins folder in assets and add core ML.NET Nuget
> binaries along with all nested
> dependencies (e.g. Sytem.Memory) and inherent native dependencies
> (e.g. CPUMathNative)
> - Use lower-level API to score ML.NET models and avoid the high level
> convinience APIs e.g.
> PredictionEngine as they currently make use of reflection emit which
> throws up.
> - Target .NET 4.x API compatibility level (default for 2018.3.4f1)
>
> Here's a unity package which shows this play.
>
>
> https://github.com/dotnet/machinelearning-samples/tree/master/samples/csharp/end-to-end-apps/Unity-HelloMLNET
>
> Does this help @mihaar <https://github.com/mihaar>?
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> <#1886 (comment)>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/Aj1kTAya0FRTJzsOj1u9P15IKc6Vwkf_ks5vKoNhgaJpZM4ZU87E>
> .
>
|
|
Hi @fashrista ! The model takes the input format of the data that it was trained with. If you train a model with your data, it will take your same input data at score time. You'll need to do two things:
Specify a training pipelinea) Load the data // Define the columns to read
var reader = mlContext.Data.CreateTextLoader(
columns: new[]
{
new TextLoader.Column("Label", DataKind.R4, 0),
new TextLoader.Column("Feature1", DataKind.R4, 1),
new TextLoader.Column("Feature2", DataKind.R4, 2),
new TextLoader.Column("Feature3", DataKind.R4, 3),
},
hasHeader: true
);
// Read the data into an IDataView
var data = reader.Read(dataFile);b) Define a learning pipeline // Concatenate the features to create a Feature vector.
// Normalize the data set so that for each feature, its maximum value is 1 while its minimum value is 0.
// Then append a linear regression trainer.
var pipeline = mlContext.Transforms.Concatenate("Features", new string[] {"Feature1", "Feature2", "Feature3"})
.Append(mlContext.Transforms.Normalize("Features"))
.Append(mlContext.BinaryClassification.Trainers.LogisticRegression());
var model = pipeline.Fit(data);c) Save the model using (var stream = File.Create(modelPath))
{
mlContext.Model.Save(model, stream);
}Load the model and score it in your appa) Load the model ITransformer loadedModel;
using (var stream = File.OpenRead(modelPath))
loadedModel = mlContext.Model.Load(stream);b) Define a class for your data and for your output public class InputData
{
public bool Label;
public float Feature1;
public float Feature2;
public float Feature3;
}
public class ScoredData
{
public bool Label;
public float Feature1;
public float Feature2;
public float Feature3;
public float Score;
public float Probability;
}c) Make a prediction engine from the model var predictionFunc = model.CreatePredictionEngine<InputData, ScoredData>(mlContext);d) Score your data my inputData = GetMyInputDataFromSomewhere();
ScoredData prediction = predictionFunc.Predict(inputData); |
|
FYI, this is the same issue as was reported for Mono (that we/Unity use). I'm looking into cherry-picking their fix: mono/mono#9033 |
I think I got some good news in a mail the other day:Re: (Case 1109657) ML.NET and Unity We have fixed this problem and it should not appear in the 2019.2.0a9 version of Unity. We highly appreciate your contribution. If you have further questions, feel free to contact us. Sincerely, I will be testing this shortly and will be reporting about results here. |
|
If you do how about uploading a link to a unity project that can be loaded and run right out of the box... to get started with.. ( a startup project). Nothing I've pulled down example wise will run. Either Missing DLL errors, or it puts Unity into some weird loop that requires you to kill unity.. running 2018.3.12f Even pulling down Ml.net from nuget directly inside Unity. doesn't compile after install... dll errors.. |
|
I will be uploading a demo with detailed instructions on how to setup a unity ML .Net project, but for now the compatible version of Unity( 2019.2.0a9) is still an alfa so, we'll have to wait a bit for it to become at least a beta. |
|
Reflection.Emit should work in the Editor, Standalone, and Android (with Mono). |
|
@joncham so sorry, my bad, shold say: ... since it calls CreateDelegate and passes a delegate type that does not have the owner type as the first argument... |
|
Look forward to any demo you can put up when it gets released into the
wild... thanks!!!!
…On Tue, Apr 9, 2019 at 2:52 AM jib fashr ***@***.***> wrote:
I will be uploading a demo with detailed instructions on how to setup a
unity ML .Net project, but for now the compatible version of Unity(
2019.2.0a9) is still an alfa so, we'll have to wait a bit for it to become
at least a beta.
Until than you can just train the models inside Unity, but you cannot make
predictions unless you find some workarround for model.predict since it
uses System.Reflection.Emit which does not comply with Unity.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#1886 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AJpiQjF_jNhYc5kdQ0TrNF3H6Vq1LCo5ks5vfDizgaJpZM4ZU87E>
.
|
|
Thanks.. looking forward to anything you can do. i'm deep diving into
learning unity/vr at the moment. ML would be a great addition. I wish
you luck going forward!!
…On Tue, Apr 9, 2019 at 2:52 AM jib fashr ***@***.***> wrote:
I will be uploading a demo with detailed instructions on how to setup a
unity ML .Net project, but for now the compatible version of Unity(
2019.2.0a9) is still an alfa so, we'll have to wait a bit for it to become
at least a beta.
Until than you can just train the models inside Unity, but you cannot make
predictions unless you find some workarround for model.predict since it
uses System.Reflection.Emit which does not comply with Unity.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#1886 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AJpiQjF_jNhYc5kdQ0TrNF3H6Vq1LCo5ks5vfDizgaJpZM4ZU87E>
.
|
|
Any luck with this? I've tried the Unity Hello World ML example and I still get the errors mentioned above on Unity 2019.2.0a13, 2019.3.0a5 and 2018.3.11f1 |
|
Ok, as promised: @DooblyNoobly @paulallenbright
This should work right out of the box ... well it did for me. Where you could mess up is when getting api files, they could be different version from mine or just some .net framework compatible. So let me also add the content of my Plugins folder (mind you all those apis may not be necesery do the cherrypicking yourselve). I hope this helps, since Unity update 19.2 I have not had any problems mentioned in previous posts in this thread. |
|
Well, I just tried this same approach and it still doenst work, Im in Unity 2019.2.0b4, and also tied with a version of 2019.3 I keep getting many errors like this: Assembly 'Assets/Plugins/Microsoft.ML.PCA.dll' will not be loaded due to errors: And yes, I did the changes that @mihaar suggests. |
|
It’s just not possible keptin’. Btw I just ended up running ML.net as a service and connecting to it from Unity via sockets. |
|
Here is a simple as pants, working unity project with ML .Net plugins: |
|
@mihaar I dont think we can reach your links above anymore. Would you mind sharing why you removed them or link to the solutions again? @gkapellmann Did you ever find a solution to this? |
|
Hello, everyone! I found workaround in Unity-HelloML.Net sample, but class Is there some way to use LoadFromEnumerable() (or another workaround)? @codemzs @rogancarr @asthana86 Can you, please, reopen this issue? |
Hi, @KrasnovPavel. Can you open a new issue describing your scenario? List what you are doing, what the expected results are, and what the actual results are. This will help us in addressing the issue. |
|
@KrasnovPavel Not sure this is a unity env thing. Have had trouble with this also outside unity env. Would sugest testing it with similar .NET framework/core version outside unity. (create a simple console app to test which.NET version is a problem) |
ghost
locked as resolved and limited conversation to collaborators
System information
Issue
Unity editor reports two errors:
1.Assembly 'Library/ScriptAssemblies/Assembly-CSharp.dll' will not be loaded due to errors:
Reference has errors 'MLAICore'.
2.Assembly 'Assets/Plugins/MLAICore.dll' will not be loaded due to errors:
Unable to resolve reference 'Microsoft.ML.Data'. Is the assembly missing or incompatible with the current platform?
I have made an API that loads model.zip file and predicts a nr given some input values(It is almost exactly as your taxi fare example). Through commenting out lines and rebuilding my api I deduced that if I put this code: MLContext ml = new MLContext(seed: 0); into any of my api methods I get this error when importing the api into unity.
It all works great but when I import this api into Unity, the unity editor returns an error.
Since unity has now support for .net 2.0 apis and frameworks 4+ I was hoping ML.Net would work in that environment. Any sugestions would be most welcom.
sampleUnityProject.zip
exampleDotnetApiProject.zip
https://stackoverflow.com/questions/53781312/ml-net-in-unity/53781387#comment94434358_53781312
Source code / logs
using System; using Microsoft.ML; namespace MLAICore { public class Class1 { public static string SayHello() { MLContext ml = new MLContext(seed: 0);//if I comment out this line code compiles in unity return "hello from core ML"; //and i get this response, if I live it as is =>err } } }Please paste or attach the code or logs or traces that would be helpful to diagnose the issue you are reporting.
The text was updated successfully, but these errors were encountered: