2.0.1 verified patch (#5605)

* update package version from 2.0.0 to 2.0.1

* Update README.md with new survey link(#5500) (#5585)

Co-authored-by: jmercado1985 <[email protected]>

* [Cherry-pick for 2.0.1 verified patch] default analytics channel (#5511) (#5584)

* update changelog from #5511

* set gym=0.20.0 #5540, update changelog

* Cap cattrs version for now(#5397)

* clean up

Co-authored-by: Vincent-Pierre BERGES <[email protected]>
Co-authored-by: Ruo-Ping Dong <[email protected]>

* Cherry picking #5283 and #5519 into 2.0.1-verified-patch to fix gRPC for mac m1 (MLA 2259) (#5602)

* Only update the native library and the c# calls that need to be made to clean up channels. (#5283)

* Fix Mac backcompat test (#5519)

Co-authored-by: Chris Goy <[email protected]>

* [Cherry-pick for 2.0.1 verified patch]Harden user PII in analytics (#5512) harden analytics (#5604)

* Harden user PII protection logic and extend TrainingAnalytics to expose detailed configuration parameters. (#5512)

* Hash128 is not a cryptographic hash, replace with HMAC-SHA256.

* Extend TrainingAnalytics side channel to expose configuration details

* Change member function scopes and hash demo_paths

* Extract tbiEvent hashing method and add test coverage

* Fixing the analytics side-channel for curriculum learning. (#5586)

* Fixing the analytics side-channel for curriculum learning.

* Made a more robust test.

* Update the changelog.

* Update com.unity.ml-agents/CHANGELOG.md

Co-authored-by: Maryam Honari <[email protected]>

Co-authored-by: Maryam Honari <[email protected]>

* update CHANGELOG

Co-authored-by: Jason Bowman <[email protected]>

Co-authored-by: jmercado1985 <[email protected]>
Co-authored-by: Vincent-Pierre BERGES <[email protected]>
Co-authored-by: Ruo-Ping Dong <[email protected]>
Co-authored-by: Jason Rupert <[email protected]>
Co-authored-by: Chris Goy <[email protected]>
Co-authored-by: Jason Bowman <[email protected]>

Author: 7 people

.yamato/training-backcompat-tests.yml

@@ -1,19 +1,19 @@
 
-test_mac_backcompat_2020.1:
-  {% capture editor_version %}2020.1{% endcapture %}
+test_mac_backcompat_2019.4:
+  {% capture editor_version %}2019.4{% endcapture %}
   {% capture csharp_backcompat_version %}1.0.0{% endcapture %}
   # This test has to run on mac because it requires the custom build of tensorflow without AVX
   # Test against 2020.1 because 2020.2 has to run against package version 1.2.0
   name: Test Mac Backcompat Training {{ editor_version }}
   agent:
     type: Unity::VM::osx
-    image: ml-agents/ml-agents-bokken-mac:0.1.4-492264
+    image: ml-agents/ml-agents-bokken-mac:0.1.5-853758
     flavor: b1.small
   variables:
     UNITY_VERSION: {{ editor_version }}
   commands:
     - |
-      python3 -m venv venv && source venv/bin/activate
+      python -m venv venv && source venv/bin/activate
       python -m pip install pyyaml --index-url https://artifactory.prd.it.unity3d.com/artifactory/api/pypi/pypi/simple
       python -m pip install unity-downloader-cli --index-url https://artifactory.prd.it.unity3d.com/artifactory/api/pypi/pypi/simple --upgrade
       unity-downloader-cli -u {{ editor_version }} -c editor --wait --fast
@@ -23,9 +23,9 @@ test_mac_backcompat_2020.1:
       python -u -m ml-agents.tests.yamato.standalone_build_tests --build-target=mac
       python -u -m ml-agents.tests.yamato.training_int_tests --csharp {{ csharp_backcompat_version }}
     - |
-      python3 -m venv venv_old && source venv_old/bin/activate
+      python -m venv venv_old && source venv_old/bin/activate
       python -m pip install pyyaml --index-url https://artifactory.prd.it.unity3d.com/artifactory/api/pypi/pypi/simple
-      python -u -m ml-agents.tests.yamato.training_int_tests --python 0.16.0
+      python -u -m ml-agents.tests.yamato.training_int_tests --python 0.24.0
   triggers:
     cancel_old_ci: true
     recurring:

README.md

@@ -174,7 +174,7 @@ package by replying to
 Your opinion matters a great deal to us. Only by hearing your thoughts on the
 Unity ML-Agents Toolkit can we continue to improve and grow. Please take a few
 minutes to
-[let us know about it](https://github.com/Unity-Technologies/ml-agents/issues/1454).
+[let us know about it](https://unitysoftware.co1.qualtrics.com/jfe/form/SV_55pQKCZ578t0kbc).
 
 For any other questions or feedback, connect directly with the ML-Agents team at
 [email protected].

com.unity.ml-agents.extensions/package.json

@@ -5,7 +5,7 @@
   "unity": "2019.4",
   "description": "A source-only package for new features based on ML-Agents",
   "dependencies": {
-    "com.unity.ml-agents": "2.0.0",
+    "com.unity.ml-agents": "2.0.1",
     "com.unity.modules.physics": "1.0.0"
   }
 }

com.unity.ml-agents/CHANGELOG.md

@@ -5,6 +5,20 @@ All notable changes to this package will be documented in this file.
 The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
 and this project adheres to
 [Semantic Versioning](http://semver.org/spec/v2.0.0.html).
+## [2.0.1] - 2021-10-13
+### Minor Changes
+#### com.unity.ml-agents / com.unity.ml-agents.extensions (C#)
+- Upgrade to 2.0.1
+- Update gRPC native lib to universal for arm64 and x86_64. This change should enable ml-agents usage on mac M1 (#5283, #5519)
+#### ml-agents / ml-agents-envs / gym-unity (Python)
+- Set gym version in gym-unity to gym release 0.20.0(#5540)
+- Harden user PII protection logic and extend TrainingAnalytics to expose detailed configuration parameters. (#5512)
+- Added minimal analytics collection to LL-API (#5511)
+
+### Bug Fixes
+#### com.unity.ml-agents / com.unity.ml-agents.extensions (C#)
+- Fixed the bug where curriculum learning would crash because of the incorrect run_options parsing. (#5586)
+#### ml-agents / ml-agents-envs / gym-unity (Python)
 
 ## [2.0.0] - 2021-09-01
 ### Minor Changes

com.unity.ml-agents/Plugins/ProtoBuffer/runtimes/osx/native/libgrpc_csharp_ext.x64.bundle

@@ -107,7 +107,7 @@ public class Academy : IDisposable
         /// Unity package version of com.unity.ml-agents.
         /// This must match the version string in package.json and is checked in a unit test.
         /// </summary>
-        internal const string k_PackageVersion = "2.0.0";
+        internal const string k_PackageVersion = "2.0.1";
 
         const int k_EditorTrainingPort = 5004;
 

com.unity.ml-agents/Runtime/Academy.cs

@@ -1,19 +1,45 @@
 using System;
+using System.Text;
+using System.Security.Cryptography;
 using UnityEngine;
 
 namespace Unity.MLAgents.Analytics
 {
+
     internal static class AnalyticsUtils
     {
+        /// <summary>
+        /// Conversion function from byte array to hex string
+        /// </summary>
+        /// <param name="array"></param>
+        /// <returns>A byte array to be hex encoded.</returns>
+        private static string ToHexString(byte[] array)
+        {
+            StringBuilder hex = new StringBuilder(array.Length * 2);
+            foreach (byte b in array)
+            {
+                hex.AppendFormat("{0:x2}", b);
+            }
+            return hex.ToString();
+        }
+
         /// <summary>
         /// Hash a string to remove PII or secret info before sending to analytics
         /// </summary>
-        /// <param name="s"></param>
-        /// <returns>A string containing the Hash128 of the input string.</returns>
-        public static string Hash(string s)
+        /// <param name="key"></param>
+        /// <returns>A string containing the key to be used for HMAC encoding.</returns>
+        /// <param name="value"></param>
+        /// <returns>A string containing the value to be encoded.</returns>
+        public static string Hash(string key, string value)
         {
-            var behaviorNameHash = Hash128.Compute(s);
-            return behaviorNameHash.ToString();
+            string hash;
+            UTF8Encoding encoder = new UTF8Encoding();
+            using (HMACSHA256 hmac = new HMACSHA256(encoder.GetBytes(key)))
+            {
+                Byte[] hmBytes = hmac.ComputeHash(encoder.GetBytes(value));
+                hash = ToHexString(hmBytes);
+            }
+            return hash;
         }
 
         internal static bool s_SendEditorAnalytics = true;

com.unity.ml-agents/Runtime/Analytics/AnalyticsUtils.cs

@@ -156,6 +156,7 @@ internal struct TrainingEnvironmentInitializedEvent
         public string TorchDeviceType;
         public int NumEnvironments;
         public int NumEnvironmentParameters;
+        public string RunOptions;
     }
 
     [Flags]
@@ -188,5 +189,6 @@ internal struct TrainingBehaviorInitializedEvent
         public string VisualEncoder;
         public int NumNetworkLayers;
         public int NumNetworkHiddenUnits;
+        public string Config;
     }
 }

com.unity.ml-agents/Runtime/Analytics/Events.cs

@@ -156,7 +156,7 @@ IList<IActuator> actuators
             var inferenceEvent = new InferenceEvent();
 
             // Hash the behavior name so that there's no concern about PII or "secret" data being leaked.
-            inferenceEvent.BehaviorName = AnalyticsUtils.Hash(behaviorName);
+            inferenceEvent.BehaviorName = AnalyticsUtils.Hash(k_VendorKey, behaviorName);
 
             inferenceEvent.BarracudaModelSource = barracudaModel.IrSource;
             inferenceEvent.BarracudaModelVersion = barracudaModel.IrVersion;

com.unity.ml-agents/Runtime/Analytics/InferenceAnalytics.cs

@@ -192,8 +192,21 @@ internal static string ParseBehaviorName(string fullyQualifiedBehaviorName)
             return fullyQualifiedBehaviorName.Substring(0, lastQuestionIndex);
         }
 
+        internal static TrainingBehaviorInitializedEvent SanitizeTrainingBehaviorInitializedEvent(TrainingBehaviorInitializedEvent tbiEvent)
+        {
+            // Hash the behavior name if the message version is from an older version of ml-agents that doesn't do trainer-side hashing.
+            // We'll also, for extra safety, verify that the BehaviorName is the size of the expected SHA256 hash.
+            // Context: The config field was added at the same time as trainer side hashing, so messages including it should already be hashed.
+            if (tbiEvent.Config.Length == 0 || tbiEvent.BehaviorName.Length != 64)
+            {
+                tbiEvent.BehaviorName = AnalyticsUtils.Hash(k_VendorKey, tbiEvent.BehaviorName);
+            }
+
+            return tbiEvent;
+        }
+
         [Conditional("MLA_UNITY_ANALYTICS_MODULE")]
-        public static void TrainingBehaviorInitialized(TrainingBehaviorInitializedEvent tbiEvent)
+        public static void TrainingBehaviorInitialized(TrainingBehaviorInitializedEvent rawTbiEvent)
         {
 #if UNITY_EDITOR && MLA_UNITY_ANALYTICS_MODULE
             if (!IsAnalyticsEnabled())
@@ -202,6 +215,7 @@ public static void TrainingBehaviorInitialized(TrainingBehaviorInitializedEvent
             if (!EnableAnalytics())
                 return;
 
+            var tbiEvent = SanitizeTrainingBehaviorInitializedEvent(rawTbiEvent);
             var behaviorName = tbiEvent.BehaviorName;
             var added = s_SentTrainingBehaviorInitialized.Add(behaviorName);
 
@@ -211,9 +225,7 @@ public static void TrainingBehaviorInitialized(TrainingBehaviorInitializedEvent
                 return;
             }
 
-            // Hash the behavior name so that there's no concern about PII or "secret" data being leaked.
             tbiEvent.TrainingSessionGuid = s_TrainingSessionGuid.ToString();
-            tbiEvent.BehaviorName = AnalyticsUtils.Hash(tbiEvent.BehaviorName);
 
             // Note - to debug, use JsonUtility.ToJson on the event.
             // Debug.Log(
@@ -236,7 +248,7 @@ IList<IActuator> actuators
             var remotePolicyEvent = new RemotePolicyInitializedEvent();
 
             // Hash the behavior name so that there's no concern about PII or "secret" data being leaked.
-            remotePolicyEvent.BehaviorName = AnalyticsUtils.Hash(behaviorName);
+            remotePolicyEvent.BehaviorName = AnalyticsUtils.Hash(k_VendorKey, behaviorName);
 
             remotePolicyEvent.TrainingSessionGuid = s_TrainingSessionGuid.ToString();
             remotePolicyEvent.ActionSpec = EventActionSpec.FromActionSpec(actionSpec);

com.unity.ml-agents/Runtime/Analytics/TrainingAnalytics.cs

@@ -501,6 +501,7 @@ internal static TrainingEnvironmentInitializedEvent ToTrainingEnvironmentInitial
                 TorchDeviceType = inputProto.TorchDeviceType,
                 NumEnvironments = inputProto.NumEnvs,
                 NumEnvironmentParameters = inputProto.NumEnvironmentParameters,
+                RunOptions = inputProto.RunOptions,
             };
         }
 
@@ -530,6 +531,7 @@ internal static TrainingBehaviorInitializedEvent ToTrainingBehaviorInitializedEv
                 VisualEncoder = inputProto.VisualEncoder,
                 NumNetworkLayers = inputProto.NumNetworkLayers,
                 NumNetworkHiddenUnits = inputProto.NumNetworkHiddenUnits,
+                Config = inputProto.Config,
             };
         }
 

com.unity.ml-agents/Runtime/Communicator/GrpcExtensions.cs

@@ -2,7 +2,7 @@
 #define MLA_SUPPORTED_TRAINING_PLATFORM
 #endif
 
-# if MLA_SUPPORTED_TRAINING_PLATFORM
+#if MLA_SUPPORTED_TRAINING_PLATFORM
 using Grpc.Core;
 #if UNITY_EDITOR
 using UnityEditor;
@@ -50,6 +50,7 @@ internal class RpcCommunicator : ICommunicator
 
         /// The Unity to External client.
         UnityToExternalProto.UnityToExternalProtoClient m_Client;
+        Channel m_Channel;
 
         /// <summary>
         /// Initializes a new instance of the RPCCommunicator class.
@@ -140,6 +141,7 @@ out input
                     Debug.Log($"Unexpected exception when trying to initialize communication: {ex}");
                 }
                 initParametersOut = new UnityRLInitParameters();
+                NotifyQuitAndShutDownChannel();
                 return false;
             }
 
@@ -181,6 +183,8 @@ out input
 
             UpdateEnvironmentWithInput(input.RlInput);
             initParametersOut = initializationInput.RlInitializationInput.ToUnityRLInitParameters();
+            // Be sure to shut down the grpc channel when the application is quitting.
+            Application.quitting += NotifyQuitAndShutDownChannel;
             return true;
 #else
             initParametersOut = new UnityRLInitParameters();
@@ -217,9 +221,9 @@ void UpdateEnvironmentWithInput(UnityRLInputProto rlInput)
         UnityInputProto Initialize(int port, UnityOutputProto unityOutput, out UnityInputProto unityInput)
         {
             m_IsOpen = true;
-            var channel = new Channel($"localhost:{port}", ChannelCredentials.Insecure);
+            m_Channel = new Channel($"localhost:{port}", ChannelCredentials.Insecure);
 
-            m_Client = new UnityToExternalProto.UnityToExternalProtoClient(channel);
+            m_Client = new UnityToExternalProto.UnityToExternalProtoClient(m_Channel);
             var result = m_Client.Exchange(WrapMessage(unityOutput, 200));
             var inputMessage = m_Client.Exchange(WrapMessage(null, 200));
             unityInput = inputMessage.UnityInput;
@@ -229,11 +233,24 @@ UnityInputProto Initialize(int port, UnityOutputProto unityOutput, out UnityInpu
             if (result.Header.Status != 200 || inputMessage.Header.Status != 200)
             {
                 m_IsOpen = false;
-                QuitCommandReceived?.Invoke();
+                NotifyQuitAndShutDownChannel();
             }
             return result.UnityInput;
         }
 
+        void NotifyQuitAndShutDownChannel()
+        {
+            QuitCommandReceived?.Invoke();
+            try
+            {
+                m_Channel.ShutdownAsync().Wait();
+            }
+            catch (Exception)
+            {
+                // do nothing
+            }
+        }
+
 #endregion
 
 #region Destruction
@@ -269,7 +286,7 @@ void SendCommandEvent(CommandProto command)
             {
                 case CommandProto.Quit:
                     {
-                        QuitCommandReceived?.Invoke();
+                        NotifyQuitAndShutDownChannel();
                         return;
                     }
                 case CommandProto.Reset:
@@ -456,7 +473,7 @@ UnityInputProto Exchange(UnityOutputProto unityOutput)
                 // Not sure if the quit command is actually sent when a
                 // non 200 message is received.  Notify that we are indeed
                 // quitting.
-                QuitCommandReceived?.Invoke();
+                NotifyQuitAndShutDownChannel();
                 return message.UnityInput;
             }
             catch (Exception ex)
@@ -488,7 +505,7 @@ UnityInputProto Exchange(UnityOutputProto unityOutput)
                 }
 
                 m_IsOpen = false;
-                QuitCommandReceived?.Invoke();
+                NotifyQuitAndShutDownChannel();
                 return null;
             }
         }