Skip to content

Make IActuator implement IHeuristicProvider. #5110

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Mar 15, 2021
Merged

Make IActuator implement IHeuristicProvider. #5110

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
5555ea0
Make IActuator implement IHeuristicProvider.
surfnerd Mar 13, 2021
d31953a
Update migration guide and migrating doc.
surfnerd Mar 13, 2021
df9f134
Remove import.
surfnerd Mar 13, 2021
b459af4
Fix backtick.
surfnerd Mar 13, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion com.unity.ml-agents.extensions/Runtime/Input/InputActionActuator.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ namespace Unity.MLAgents.Extensions.Input
/// <see cref="Agent"/>'s <see cref="BehaviorParameters"/> indicate that the Agent is running in Heuristic Mode,
/// this Actuator will write actions from the <see cref="InputSystem"/> to the <see cref="ActionBuffers"/> object.
/// </summary>
public class InputActionActuator : IActuator, IHeuristicProvider, IBuiltInActuator
public class InputActionActuator : IActuator, IBuiltInActuator
{
readonly BehaviorParameters m_BehaviorParameters;
readonly InputAction m_Action;
Expand Down
1 change: 1 addition & 0 deletions com.unity.ml-agents/CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ and this project adheres to
- The interface for disabling discrete actions in `IDiscreteActionMask` has changed.
`WriteMask(int branch, IEnumerable<int> actionIndices)` was replaced with
`SetActionEnabled(int branch, int actionIndex, bool isEnabled)`. See the
- IActuator now implements IHeuristicProvider. (#5110)
[Migration Guide](https://github.com/Unity-Technologies/ml-agents/blob/release_14_docs/docs/Migrating.md) for more
details. (#5060)
#### ml-agents / ml-agents-envs / gym-unity (Python)
Expand Down
4 changes: 1 addition & 3 deletions com.unity.ml-agents/Runtime/Actuators/ActuatorManager.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -244,9 +244,7 @@ public void ApplyHeuristic(in ActionBuffers actionBuffersOut)
discreteStart,
numDiscreteActions);
}

var heuristic = actuator as IHeuristicProvider;
heuristic?.Heuristic(new ActionBuffers(continuousActions, discreteActions));
actuator.Heuristic(new ActionBuffers(continuousActions, discreteActions));
continuousStart += numContinuousActions;
discreteStart += numDiscreteActions;
}
Expand Down
2 changes: 1 addition & 1 deletion com.unity.ml-agents/Runtime/Actuators/IActuator.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ namespace Unity.MLAgents.Actuators
/// <summary>
/// Abstraction that facilitates the execution of actions.
/// </summary>
public interface IActuator : IActionReceiver
public interface IActuator : IActionReceiver, IHeuristicProvider
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we still need this to be a separate interface with only one method? Can it be part of IActionReceiver?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, good point. I made IActionReceiver in order to help with backward compatibility. Let me think about this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think for now i'd like to leave it the way it is as it allows for indirection of actions. There might be an actuator that you want to write that delegates actions to other action receivers. maybe in a robotics use case or something.

{
/// <summary>
/// The specification of the actions for this IActuator.
Expand Down
2 changes: 1 addition & 1 deletion com.unity.ml-agents/Runtime/Actuators/VectorActuator.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ namespace Unity.MLAgents.Actuators
/// <summary>
/// IActuator implementation that forwards calls to an <see cref="IActionReceiver"/> and an <see cref="IHeuristicProvider"/>.
/// </summary>
internal class VectorActuator : IActuator, IHeuristicProvider, IBuiltInActuator
internal class VectorActuator : IActuator, IBuiltInActuator
{
IActionReceiver m_ActionReceiver;
IHeuristicProvider m_HeuristicProvider;
Expand Down
2 changes: 2 additions & 0 deletions docs/Migrating.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,8 @@ public override void WriteDiscreteActionMask(IDiscreteActionMask actionMask)
actionMask.SetActionEnabled(branch, 3, false);
}
```
- The `IActuator` interface now implements `IHeuristicProvider`. Please add the corresponding `Heuristic(in ActionBuffers)`
method to your custom Actuator classes.

## Migrating to Release 13
### Implementing IHeuristic in your IActuator implementations
Expand Down