Skip to content

[dart2wasm] Argument types are not checked in dynamic invocations #50367

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
osa1 opened this issue Nov 3, 2022 · 16 comments
Closed

[dart2wasm] Argument types are not checked in dynamic invocations #50367

osa1 opened this issue Nov 3, 2022 · 16 comments
Assignees
Labels
area-dart2wasm Issues for the dart2wasm compiler.

Comments

@osa1
Copy link
Member

osa1 commented Nov 3, 2022

This causes a few test failures and I was able to come up with two tiny repros, one causes Wasm trap, one crashes in compile time.

This one fails with Wasm trap:

class B {
  void aaa(double x) {}
}

void main() {
  (B() as dynamic).aaa(123);
}
wasm-function[722]:0x22959: RuntimeError: illegal cast
RuntimeError: illegal cast
    at aaa dynamic trampoline (wasm://wasm/000aa906:wasm-function[722]:0x22959)
    at main (wasm://wasm/000aa906:wasm-function[50]:0x15fa9)
    at $main (wasm://wasm/000aa906:wasm-function[51]:0x15fb0)
    at Module.invoke (/Users/omersa/dart/sdk/sdk/pkg/dart2wasm/bin/dart2wasm_runtime.mjs:410:28)
    at main (pkg/dart2wasm/bin/run_wasm.js:46:21)

If I add an optional parameter to the method then it crashes in compile time:

class B {
  void aaa(double x, [int? start]) {}
}

void main() {
  (B() as dynamic).aaa(123);
}
Exception in IntJudgment at file:///Users/omersa/dart/sdk/sdk/test.dart:18:24
Unhandled exception:
type 'NumType' is not a subtype of type 'RefType' in type cast
#0      Translator.convertType (package:dart2wasm/translator.dart:908:28)
#1      CodeGenerator.wrap (package:dart2wasm/code_generator.dart:622:18)
#2      CodeGenerator.visitArgumentsLists (package:dart2wasm/code_generator.dart:2276:7)
#3      DynamicDispatcher._callDynamic.<anonymous closure> (package:dart2wasm/dynamic_dispatch.dart:84:19)
#4      DynamicDispatcher._emitDynamicCallBody (package:dart2wasm/dynamic_dispatch.dart:182:22)
#5      DynamicDispatcher._callDynamic (package:dart2wasm/dynamic_dispatch.dart:60:26)
#6      DynamicDispatcher.emitDynamicCall (package:dart2wasm/dynamic_dispatch.dart:480:16)
#7      CodeGenerator.visitDynamicInvocation (package:dart2wasm/code_generator.dart:1499:32)
#8      DynamicInvocation.accept1 (package:kernel/ast.dart:5555:9)
#9      CodeGenerator.wrap (package:dart2wasm/code_generator.dart:621:37)
#10     CodeGenerator.visitExpressionStatement (package:dart2wasm/code_generator.dart:965:5)
#11     ExpressionStatement.accept (package:kernel/ast.dart:9303:43)
#12     CodeGenerator.visitStatement (package:dart2wasm/code_generator.dart:632:12)
#13     CodeGenerator.visitBlock (package:dart2wasm/code_generator.dart:731:7)
#14     Block.accept (package:kernel/ast.dart:9359:43)
#15     CodeGenerator.visitStatement (package:dart2wasm/code_generator.dart:632:12)
#16     CodeGenerator.generateBody (package:dart2wasm/code_generator.dart:437:7)
#17     CodeGenerator.generate (package:dart2wasm/code_generator.dart:186:12)
#18     Translator.translate (package:dart2wasm/translator.dart:459:15)
#19     compileToModule (package:dart2wasm/compile.dart:86:21)
<asynchronous suspension>
#20     main (file:///Users/omersa/dart/sdk/sdk/pkg/dart2wasm/bin/dart2wasm.dart:120:23)
@osa1 osa1 added the area-dart2wasm Issues for the dart2wasm compiler. label Nov 3, 2022
@osa1
Copy link
Member Author

osa1 commented Nov 7, 2022

Another repro:

void test(l1) {
  l1.insert(null, 5);
}

void main() {
  test([0, 1, 2, 3, 4]);
}

AOT output:

Unhandled exception:
type 'Null' is not a subtype of type 'int' of 'index'
#0      testModifiableList (file:///usr/local/google/home/omersa/dart/sdk/sdk/test.dart:2:6)
#1      main (file:///usr/local/google/home/omersa/dart/sdk/sdk/test.dart:6:3)
#2      _delayEntrypointInvocation.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:297:19)
#3      _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:192:12)

dart2wasm traps:

wasm-function[728]:0x22baa: RuntimeError: dereferencing a null pointer
RuntimeError: dereferencing a null pointer
    at insert dynamic trampoline (wasm://wasm/000ac746:wasm-function[728]:0x22baa)
    at testModifiableList (wasm://wasm/000ac746:wasm-function[727]:0x22b25)
    at main (wasm://wasm/000ac746:wasm-function[53]:0x160ff)
    at main tear-off (wasm://wasm/000ac746:wasm-function[54]:0x16105)
    at _invokeMain (wasm://wasm/000ac746:wasm-function[64]:0x163d7)
    at Module.invoke (/usr/local/google/home/omersa/dart/sdk/sdk/pkg/dart2wasm/bin/dart2wasm_runtime.mjs:410:28)
    at main (/usr/local/google/home/omersa/dart/sdk/sdk/pkg/dart2wasm/bin/run_wasm.js:46:21)

@osa1
Copy link
Member Author

osa1 commented Nov 14, 2022

My plan to fix this:

  • For every member that can be called in a dynamic invocation we will create a new Reference that checks the argument types and calls the actual method.

  • These references need to be in the dispatch table, because every override of a method will need a different type checking code. For example:

    class A {}
    class B extends A {}
    
    class C1 {
      void f(B b) {}  
    }
    
    class C2 extends C1 {
      @override
      void f(A a) {}
    }
    
    void main() {
      // Runtime type check fails
      dynamic x = C1();
      x.f(A());
    
      // Runtime type check passes
      dynamic y = C2();
      y.f(A());
    }

    So the selector for f will have different dynamic invocation targets for each override of f, and in the invocation site we select the right one similar to virtual calls: dispatchTable[classId + dynamicInvocationOffset].

  • Mapping a member to the dynamicInvocationSelectorOffset used above is already handled by DispatchTable.selectorsForDynamicNode.

  • When populating the dynamicGets, dynamicSets, and dynamicInvocations maps (used by DispatchTable.selectorsForDynamicNode), we create a new reference, instead of using the existing reference for the non-dynamic invocation of the member.

  • When generating code for a reference (CodeGenerator.generate) we need to check if the reference is for a dynamic invocation target so that we can generate type checks.

    This is handled similarly to how we check for tear-off references: we add a new expando in reference_extensions.dart with accessors. The expando will be populated as we create new references for dynamic invocations in DispatchTable.build.

  • The code for a dynamic invocation target does the type checks and calls the instance invocation target.

    Generating type checks should be straightforward as we do it in other places in the code generator.

  • When generating the code for a dynamic invocation we already know the class ID of the receiver and the instance invocation target's table offset. So after the type check we just generate a call_indirect with a constant as offset.

    (I think we should be able to generate a direct call here?)

I think that's it. Any feedback before I start implementing this @askeksa-google @joshualitt?

@askeksa-google
Copy link

If you are implementing this as part of the existing dynamic call mechanism, then we should be able to always emit a direct call, so it's not necessary to put the type checking trampoline into the dispatch table.

The existing dynamic call code already has a concept of emitting trampolines to share code between calls of the same selector in most cases. Would it be possible to reuse that mechanism for the type checking trampolines (i.e. do the type check inside the existing trampoline) instead of adding a separate kind of reference? In cases where we can't use a trampoline, the checks can be performed inline in the call like the existing checks. Basically this amounts to just adding the type checks to the call code wherever that is emitted.

@osa1
Copy link
Member Author

osa1 commented Nov 15, 2022

Would it be possible to reuse that mechanism for the type checking trampolines (i.e. do the type check inside the existing trampoline) instead of adding a separate kind of reference? In cases where we can't use a trampoline, the checks can be performed inline in the call like the existing checks. Basically this amounts to just adding the type checks to the call code wherever that is emitted.

Hm, right, I think this should be possible and it'll be simpler too. I'll implement this.

I was assuming we want to avoid duplicating type checking code every time a member is called in dynamic invocation (emitTypeTest looks scary), but maybe that's OK.

@osa1 osa1 self-assigned this Nov 15, 2022
@osa1
Copy link
Member Author

osa1 commented Nov 21, 2022

For the type checks I was hoping to use makeType to push the _Type object of a member parameter onto the stack, to be passed to _isSubtype, but that function wants to resolve type parameters using the current class or function's type arguments. In dynamic invocations we want to resolve type parameters using the receiver's type arguments. For example:

class C<A> {
  A _a;
  
  C(this._a);
  
  void setA(A a) {
    _a = a;
  }
}

void main() {
  dynamic a = C<int>(123);
  a.setA(123);
}

In a.setA(...), A needs to be resolved using a's type arguments. makeType wants to resolve it using current class (which doesn't exist in this example since the call site is not in a class) or current function's (main) type arguments.

To allow specifying how a type parameter should be resolved in makeType I refactored it and added a parameter:

w.ValueType makeType(CodeGenerator codeGen, DartType type,
    void instantiateTypeParameter(TypeParameter parameter)) { ... }

This works fine but reveals another problem: makeType generates code in the current function being compiled in CodeGenerator, but when generating a trampoline we create a new function and generate instructions in that function. CodeGenerator cannot be used in when generating code in a trampoline function.

So we need to decouple makeType from CodeGenerator if we want to use it when generating trampolines. We could still have a CodeGenerator.makeType method that works as before, calling the more general makeType that generates in a given function.

@askeksa-google
Copy link

I think your idea of generating type testing stubs via a special kind of reference would solve this. Then the generation of that stub can set up the proper environment for this and the type arguments in the code generator like it's done for ordinary bodies, async wrappes and lambdas. It would still not need to go in the dispatch table, since the calls from the dynamic calls/trampolines can be direct.

This is somewhat similar to how the VM does it, actually. It also has special stubs in front of methods to type check arguments for dynamic calls.

@osa1
Copy link
Member Author

osa1 commented Nov 21, 2022

I think your idea of generating type testing stubs via a special kind of reference would solve this ...

Good point.

I tried decoupling makeType and CodeGenerator. The main difficulty seems to be this code when creating a function type:

List<Expression> expressions = [];
for (NamedType n in type.namedParameters) {
expressions.add(_isTypeConstant(n.type)
? ConstantExpression(
translator.constants.makeNamedParameterConstant(n),
namedParameterType)
: ConstructorInvocation(
namedParameterConstructor,
Arguments([
StringLiteral(n.name),
TypeLiteral(n.type),
BoolLiteral(n.isRequired)
])));
}
w.ValueType namedParametersListType =
codeGen.makeListFromExpressions(expressions, namedParameterType);

This allocates parameter list for the function type, and the list elements are compiled from expressions. So makeType needs to be able to compile expressions.

I think it would probably be easier to refactor CodeGenerator to allow generating code in a different DefinedFunction than the initial one. State like break labels, finalizers, mapping from function type parameters to Wasm locals etc. need to be saved and restored when overriding the current function. I think it should be doable.

Before doing that, it should simplify the code a little bit it we split CodeGenerator into multiple classes for compiling expressions, statements, and initializers. With this we will need a class for the "current function state" (since they will generate code in the same function and need that state) with labels, finalizers etc., and once we have that it should be simple to save and restore that state when generting code in a different function (when generating trampolines).

@osa1
Copy link
Member Author

osa1 commented Nov 21, 2022

I think it would probably be easier to refactor CodeGenerator to allow generating code in a different DefinedFunction than the initial one

I just realized it's actually very easy to create a new CodeGenerator. I will try creating a new CodeGenerator in _emitTrampoline and generate code using that CodeGenerator in makeType.

@askeksa-google
Copy link

It could make sense to refactor some of the code generator at some point, yes. Is any of that necessary with the type checking stub approach, though?

@osa1
Copy link
Member Author

osa1 commented Nov 21, 2022

I just realized it's actually very easy to create a new CodeGenerator. I will try creating a new CodeGenerator in _emitTrampoline and generate code using that CodeGenerator in makeType.

This works and it takes only a few lines, but the refactoring I mentioned above in makeType to take a "instantiate type parameter" argument:

w.ValueType makeType(CodeGenerator codeGen, DartType type,
    void instantiateTypeParameter(TypeParameter parameter)) { ... }

is a bit difficult to do because makeType is sometimes called recursively with unrelated code generator functions like "wrap" between the recursive calls. So instantiateTypeParameter argument needs to be passed to most (if not all) code generator methods, or needs to be made a field.

Example call stack:

...
#2      Types.makeType (package:dart2wasm/types.dart:472:31)
#3      CodeGenerator.visitTypeLiteral (package:dart2wasm/code_generator.dart:2518:18)
#4      TypeLiteral.accept1 (package:kernel/ast.dart:8371:9)
#5      CodeGenerator.wrap (package:dart2wasm/code_generator.dart:621:37)
#6      CodeGenerator.makeListFromExpressions.<anonymous closure> (package:dart2wasm/code_generator.dart:2456:47)
#7      CodeGenerator.makeList (package:dart2wasm/code_generator.dart:2439:21)
#8      CodeGenerator.makeListFromExpressions (package:dart2wasm/code_generator.dart:2453:7)
#9      Types._makeTypeList (package:dart2wasm/types.dart:357:36)
#10     Types._makeInterfaceType (package:dart2wasm/types.dart:370:5)
#11     Types.makeType (package:dart2wasm/types.dart:489:7)
#12     DynamicDispatcher._emitTrampoline.<anonymous closure> (package:dart2wasm/dynamic_dispatch.dart:450:14)
...

At the bottom _emitTrampoline passes the right instantiateTypeParameter argument, but code generator then calls makeType again with the original initialTypeParameter that resolves the parameter using this. In the middle there's wrap and a visitor method, which need to be aware of instantiateTypeParameter argument and pass it along.

We could try making instantiateTypeParameter a field, but trying to override it feels hacky either way.

Maybe the stub approach is the best.

@osa1
Copy link
Member Author

osa1 commented Nov 22, 2022

I realized as I read the code a bit more that generating type-checking wrapper functions (AOT calls these "dynamic invocation forwarders") is actually much simpler than my original plan above.

First, (not quite a blocker for this issue but a simplification) we don't need dispatch table or selectors to compile dynamic invocations. We should decouple dispatch table and dynamic invocation code. I will do that first.

(I suspect the reason why we used dispatch table was because we thought dynamic calls need to go through the dispatch table in runtime, but that's not the case. I'm updating indirect calls via the dispatch table with direct calls in https://dart-review.googlesource.com/c/sdk/+/271322. Another CL in progress to remove rest of the uses of selectors and dispatch table.)

After that, we add a pass before code generation for members to create new references for dynamic invocation forwarders for members, using the methodOrSetterCalledDynamically metadata. Mapping members to forwarders and checking if a member being compiled is a forwarder will be done the way described in my original plan (with an expando).

The rest should be trivial. When compiling a forwarder we generate type checks using makeType and _isSubType. I have this part working in https://dart-review.googlesource.com/c/sdk/+/270641 and then generate a direct call to the member. (reference.asMember.enclosingClass should give the target class)

@askeksa-google
Copy link

Sounds like a good plan! A few thoughts...

After that, we add a pass before code generation for members to create new references for dynamic invocation forwarders for members, using the methodOrSetterCalledDynamically metadata.

I would expect that a separate pass is not needed here. The stubs can be requested lazily by the dynamic call generation by passing the dynamic stub reference to functions.getFunction. I think this will work better together with tree shaking.

The rest should be trivial. When compiling a forwarder we generate type checks using makeType and _isSubType. I have this part working in https://dart-review.googlesource.com/c/sdk/+/270641 and then generate a direct call to the member. (reference.asMember.enclosingClass should give the target class)

Use emitTypeTest for the type tests. This will make sure that future optimizations to type testing (which will be handled by emitTypeTest) will also apply to these dynamic checks.

@osa1
Copy link
Member Author

osa1 commented Nov 24, 2022

I would expect that a separate pass is not needed here. The stubs can be requested lazily by the dynamic call generation by passing the dynamic stub reference to functions.getFunction. I think this will work better together with tree shaking.

We discussed this offline but just to document it here, the pass is to map a member name to the forwarder function that will check the class ID, then check the arguments type, and then call the actual member.


In https://github.com/osa1/sdk/tree/dart2wasm_forwarders I have a prototype that again blocked by makeType (or other functions that use makeType, like emitTypeTest).

In that branch I wanted to avoid adding a new pass and I kept using DispatchTable.build to collect members that are dynamically invoked.

For each member that can be dynamically accessed (in a dynamic set, get, or invocation) I generate a function that checks receiver class ID with the target class IDs, and when a match is found I generate the shape check (number of arguments must match) and then type checks for the arguments before calling the member.

The problem is these forwarder functions are not class members, so CodeGenerator cannot be used during code generation for those, and as a result I can't emit type checks for the reasons described in earlier comments.

So what we really want to do is, we still want functions for dynamic invocations of members, but type checks need to be done by instance methods of the receivers, which will have access to this, and know about class-bound type parameters.

For example, suppose we have this class:

class C<A> {
  void f<B>(A a, B b) { ... }
}

and this dynamic invocation:

(C<int>() as dynamic).f<bool>(true, "test")

The type tests will compare the argument type A with bool, and B with String. For that we need to be able to resolve A and B. The easiest way to do is to call a member of the same receiver (the receiver in the dynamic invocation) which does the type checking. That way we can emit a type test in the method body with emitTypeTest and it will be able to resolve A.

For B, I think we somehow need to adjust CodeGenerator.typeLocals so that makeType (or emitTypteTest etc.) will resolve B to the actual type argument. I'm not 100% sure on how to do this. Basically we need to adjust CodeGenerator state so that instantiateTypeParameter will resolve B to the actual type argument.

Where to compare the call site and member shapes? When we find the class matching the receiver's class ID, we need to check that the member and the call sites have compatible "shapes", i.e. it should have required positional parameters, should either have no type args or the same number of type args as the member, it should have required named arguments and no extra named arguments.

This check can be done in the forwarder or in the type-checking members. The code should not be too complicated, so we won't need CodeGenerator. Any complicated parts can be implemented in Dart.

Note that if we implement shape checking in the forwarder, then the type-checking methods can take type arguments, instead of taking a Wasm array of types representing the type arguments, but this probably won't simplify the type checks. In the example above, B is bound in the member f's type, and we won't have the binder in the body of the type-checking method even if it takes a type argument.

So for now I'm thinking of implementing simple forwarders and check everything (shape and types) in instance members.

@osa1
Copy link
Member Author

osa1 commented Nov 26, 2022

I almost finished implementing the plan above in https://github.com/osa1/sdk/tree/dart2wasm_forwarders_2. Type checking of named arguments implemented as well. However type checking with type parameters do not work as expected yet. It needs a bit more debugging..

@osa1
Copy link
Member Author

osa1 commented Nov 26, 2022

Type checking works nicely as well. I just need to implement dynamic sets and then we can start testing and refactoring.

@osa1
Copy link
Member Author

osa1 commented Nov 27, 2022

osa1 added a commit to osa1/sdk that referenced this issue Dec 5, 2022
This reimplements dynamic call code generation to add support for type
checking, named parameters (optional and required), and fixes a few
related bugs on the way.

Currently we do not try to be as efficient as possible. The goal with
this patch to implement it correctly.

Summary of the changes:

- For every dynamic access kind and member name, we generate a new
  "forwarder" function. Dynamic gets, sets, and invocations are compiled
  to calls to the forwarders with the right access kind (invocation,
  get, set) and member name.

  For example, if the program has dynamic invocation of a member "f", we
  create an "invocation forwarder for f". If it has a dynamic get of a
  member "x", we generate "getter forwarder for x".

- Forwarder functions take 4 arguments:

  - Receiver of the invocation, get, or set.

  - A Dart list for type arguments in the invocation. For gets and sets
    the list is empty.

  - A Dart list for positional arguments in the invocation. For gets the
    list is empty. For sets, the list only has one element.

  - A Dart list for named arguments. For gets and sets the list is
    empty. The list has alternating elements of type `Symbol` and
    `Object?`, for the name and value of the named parameters.

- A forwarder function compares receiver class ID with the potential
  targets of the call. When it finds a match, if compares the callee
  "shape" with the parameters passed in the call site.

  As it compares the shapes it adjusts argument lists:

  - Creates default values for missing optional and named arguments

  - Reorders the named argument list to match order expected by the
    callee

  If it can't find a matching class ID and a member with the right name
  and shape, it calls `noSuchMethod` on the receiver.

  If it finds a matching class ID and a member, it calls the "type
  checker" for the member, passing the original receiver and adjusted
  argument lists.

- A "type checker" implements argument type checking for a member, and
  it's a member of the same class as the member it's checking types
  for. This is to allow accessing class-bound type parameters when
  generating type checking code.

- Type checking is implemented using `_isSubtype` on arguments in the
  lists.

- When type checking is successful a type checker calls the original
  member, passing the arguments as expected by the member.

  If type checking is unsuccessful it throws a type error.

Most of the changes are for generating Wasm functions that compare
shapes, adjusts argument lists, and checks types.

Changes to members:

- `Translator.dynamics` fields is renamed to `Translator.forwarders`

- New field `Translator.forwarderFunctionType` added for the Wasm
  function type of forwarder and type checker functions.

- Two new code gen utilities added:
  - `Translator.indexArray`: generates code that indexes a Dart array
  - `Translator.getArrayLength`: generates code that gets length of a
    Dart array

- New `Reference` extensions added to get type checker function
  reference of members

- New library `named_parameters` implements two helper functions for
  dealing with named argument lists

- The library `dynamic_dispatch` is rewritten and now consists of two
  classes:

  - `Forwarders`: maintains mapping from call kind (get, set,
    invocation) and member name to forwarder functions.

  - `Forwarder`: a single forwarder, implements code generation for
    forwarder functions.

- `CodeGenerator` gets 3 new members:

  - `_callForwader` generates call to a forwarder

  - `generateSetterTypeCheckerMethod` generates code for a type checker
    of a setter function.

  - `generateInvocationTypeCheckerMethod` generates code for a type
    checker of a method.

    (TODO: This should be renamed to `generateMethodTypeCheckerMethod`
    for consistency)

Fixes dart-lang#50367

Change-Id: I2b9d84237c8517bd217166d8acb67e025f0498fb
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-dart2wasm Issues for the dart2wasm compiler.
Projects
None yet
Development

No branches or pull requests

2 participants