Skip to content

Latest commit

 

History

History
1620 lines (1240 loc) · 70 KB

feature-specification.md

File metadata and controls

1620 lines (1240 loc) · 70 KB

Augmentations

Author: [email protected], [email protected], [email protected]
Version: 1.35 (see Changelog at end)

Augmentations allow spreading your implementation across multiple locations, both within a single file and across multiple files. They can add new top-level declarations, inject new members into classes, and wrap functions and variables in additional code.

Motivation

Dart libraries are the unit of code reuse. When an API is too large to fit into a single file, you can usually split it into multiple libraries and then have one main library export the others. That works well when the functionality in each file is made of separate top-level declarations.

However, sometimes a single class declaration is too large to fit comfortably in a file. Dart libraries and even part files are no help there. Because of this, users have asked for something like partial classes in C# (#252 71 👍, #678 18 👍). C# also supports splitting the declaration and implementation of methods into separate files.

Generated code

Size isn't the only reason to split a library into multiple files. Code generation is common in Dart. AngularDart compiles HTML templates to Dart files. The freezed and built_value packages generate Dart code to implement immutable data structures.

In cases like this, it's important to have the hand-authored and machine-generated code in separate files so that the code generator doesn't inadvertently erase a user's code. AngularDart generates a separate library for the component. The freezed and built_value packages generate part files.

Macros

The language team is investigating adding macros to Dart. In theory, this would reduce the need for text-based code generation language features. Instead of code generation, users could write macros instead. They'd let the compiler expand them, and not worry about anything ending up on disk.

If users never needed to fix bugs in macros, look at stack traces, step into code in a debugger, or navigate to the location of a compile error, that would be true. But understanding and digging into code is fundamental to programming. An error could be thrown from code generated by a macro. A macro might generate code that produces a compile error (either deliberately because the macro was misused or because the macro itself has a bug). A user might be profiling their application and a macro might generate particularly slow code.

In all of those cases, users need some way to read and understand code produced by macros. So, even if that code is generated automatically by the compiler as part of macro expansion, it's still useful to have it be in a canonical well-specified form that users can understand.

This proposal defines that format. The idea is that a Dart compiler executes macros and then produces one or more new part files that contain all of the changes that the macros made to the library where they are applied, as new declarations to be added or augmentations that modify existing declarations. The compiler then adds those part files to the existing libraries.

But improved part files and augmenting declarations are not only a serialization format for macros. They are first-class language features that can be produced by non-macro code generators or written by hand by users who simply want to break a giant library or class into smaller files.

Augmentation declarations

This feature introduces the modifier augment as the first token of many kinds of declarations. These declarations are known as augmentation declarations.

In Dart without this feature there are no augmentation declarations. Now that we are adding augmentation declarations we need to have a term that denotes a declaration which is not an augmentation. That is, it is one of the "normal" declarations that we've had all the time.

We say that a declaration which is not an augmentation declaration is an introductory declaration.

Augmentation declarations include:

  • Type augmentations, which can add new members to types, including adding new values to enums, or even alter the type hierarchy by adding mixin applications to a class.

  • Function augmentations, which can replace the body of a function, or provide a body if none was present.

  • Variable augmentations, which can wrap the initializer of a variable in the augmented library, or provide an initializer if none was present.

A non-local variable induces a getter and possibly a setter. It is possible to augment getters and setters (this is a kind of function augmentation), including the ones that are induced by variables.

This means that augmentation of a variable declaration can be a getter declaration that augments the induced getter, a setter declaration that augments the induced setter, or a variable declaration that augments the initializer. The augmenting declarations can themselves be an implicitly induced getter or setter, or an explicitly declared one.

Note that an abstract variable declaration and an external variable declaration correspond to a getter declaration and possibly a setter declaration. There is no notion of an initializing expression, and hence also no support for augmenting it.

These operations cannot be expressed today using only imports, exports, or part files. Any Dart file (library file or part file) can contain augmentation declarations. In particular, an augmentation can augment a declaration in the same file in which it occurs.

A type augmentation can add new members to an existing type, or augment a member declaration in the same context (that is, in the same type augmentation, or in a type declaration that it augments).

Because of augmentations, non-abstract class, mixin, mixin class, enum, extension type, and extension declarations are now allowed to contain abstract member declarations, as long as those members are equipped with a body by an augmentation declaration.

Augmented expressions

An augmentation that replaces the body of a function may also want to preserve and run the code of the augmented declaration (hence the name "augmentation"). It may want to run its own code before the augmented code, after it, or both. To support that, we allow a new expression syntax inside the "bodies" of augmenting declarations (some function bodies and variable initializers). Inside an expression in an augmenting member declaration, the identifier augmented can be used to refer to the augmented function, getter, or setter body, or variable initializer. This is a contextual reserved word within augment declarations, and has no special meaning outside of that context. See the augmented expression section for a full specification of what augmented means, and how it must be used, in the various contexts.

Note that within an augmenting member declaration, a reference to a member by the same name refers to the final version of the member (and not the one being augmented). The only way to reference the augmented member is by using the keyword augmented.

Syntax

The grammar changes are fairly simple. The grammar is modified to allow an augment modifier before various declarations:

topLevelDeclaration ::= classDeclaration
  | mixinDeclaration
  | extensionTypeDeclaration
  | extensionDeclaration
  | enumType
  | typeAlias
  | 'augment'? 'external' functionSignature ';'
  | 'augment'? 'external' getterSignature ';'
  | 'augment'? 'external' setterSignature ';'
  | 'augment'? 'external' finalVarOrType identifierList ';'
  | 'augment'? functionSignature (functionBody | ';')
  | 'augment'? getterSignature (functionBody | ';')
  | 'augment'? setterSignature (functionBody | ';')
  | 'augment'? ('final' | 'const') type? initializedIdentifierList ';'
  | 'augment'? 'late' 'final' type? initializedIdentifierList ';'
  | 'augment'? 'late'? varOrType initializedIdentifierList ';'

classDeclaration ::= 'augment'? (classModifiers | mixinClassModifiers)
    'class' typeWithParameters superclass? interfaces?
    memberedDeclarationBody
  | 'augment'? classModifiers 'mixin'? 'class' mixinApplicationClass

mixinDeclaration ::= 'augment'? 'base'? 'mixin' typeIdentifier
  typeParameters? ('on' typeNotVoidNotFunctionList)? interfaces?
  memberedDeclarationBody

extensionDeclaration ::=
    'extension' typeIdentifierNotType? typeParameters? 'on' type
    memberedDeclarationBody
  | 'augment' 'extension' typeIdentifierNotType typeParameters?
    memberedDeclarationBody

extensionTypeDeclaration ::=
    'extension' 'type' 'const'? typeIdentifier
    typeParameters? representationDeclaration interfaces?
    memberedDeclarationBody
  | 'augment' 'extension' 'type' typeIdentifier typeParameters? interfaces?
    memberedDeclarationBody

enumType ::= 'augment'? 'enum' typeIdentifier
  typeParameters? mixins? interfaces?
  '{' enumEntry (',' enumEntry)* (',')?
  (';' memberDeclarations)? '}'

typeAlias ::= 'augment'? 'typedef' typeIdentifier typeParameters? '=' type ';'
  | 'augment'? 'typedef' functionTypeAlias

memberedDeclarationBody ::= '{' memberDeclarations '}'

memberDeclarations ::= (metadata memberDeclaration)*

memberDeclaration ::= 'augment'? declaration ';'
  | 'augment'? methodSignature functionBody

enumEntry ::= metadata 'augment'? identifier argumentPart?
  | metadata 'augment'? identifier typeArguments?
    '.' identifierOrNew arguments

declaration ::= 'external'? factoryConstructorSignature
  | 'external' constantConstructorSignature
  | 'external' constructorSignature
  | 'external'? 'static'? getterSignature
  | 'external'? 'static'? setterSignature
  | 'external'? 'static'? functionSignature
  | 'external' ('static'? finalVarOrType | 'covariant' varOrType) identifierList
  | 'external'? operatorSignature
  | 'abstract' (finalVarOrType | 'covariant' varOrType) identifierList
  | 'static' 'const' type? initializedIdentifierList
  | 'static' 'final' type? initializedIdentifierList
  | 'static' 'late' 'final' type? initializedIdentifierList
  | 'static' 'late'? varOrType initializedIdentifierList
  | 'covariant' 'late' 'final' type? identifierList
  | 'covariant' 'late'? varOrType initializedIdentifierList
  | 'late'? 'final' type? initializedIdentifierList
  | 'late'? varOrType initializedIdentifierList
  | redirectingFactoryConstructorSignature
  | constantConstructorSignature (redirection | initializers)?
  | constructorSignature (redirection | initializers)?

It is a compile-time error if:

  • A declaration marked augment is also marked external. (TODO: Probably remove for functions, so change to "A variable declaration". A macro should be able to implement a method as an external with a @JS() annotation.)

Static semantics

Declaration ordering relations

As part of the meta-programming and augmentation features, we expand the capabilities of part files. See "Parts with Imports".

With that feature, a part file can now have its own import and export directives, and further nested part files, with part files inheriting the imports and prefixes of their parent (part or library) file.

Augmentation declarations interact with part files mainly in restrictions on where an augmenting declaration may occur relative to the declaration it augments, as described below.

For this, we define the following relations on declarations based on the relations between files of a library.

We say that a syntactic declaration occurs in a Dart file if the declaration's source code occurs in that Dart file.

We then say that a Dart file contains a declaration if the declaration occurs in the file itself, or if any of the files included by the Dart file contain the declaration. That is, if the declaration occurs in a file in the subtree of that Dart file.

We then define two orderings of declarations in a library, one partial and one complete, as follows:

We define a partial ordering on syntactic declarations of a library, is above, such that a syntactic declaration A is above a syntactic declaration B if and only if:

  • A and B occur in the same file, and the start of A is syntactically before the start of B, in source order, or
  • The file where A occurs includes the file where B occurs.

We define a total ordering relation (transitive, anti-symmetric, irreflexive) on declarations of a library, is before (and its reverse, is after) such that for any two syntactic declarations A, and B:

  • If A and B occur in the same file, then:
    • If the start of A is syntactically before the start of B in source order, then A is before B.
    • Otherwise B is before A.
  • Otherwise A and B occur in different files:
    • Let F be the least containing file for those two files.
    • If A occurs in F then A is before B.
    • If B occurs in F then B is before A.
    • Otherwise A and B are contained in distinct included files of F.
    • If the part directive in F including the file that contains A is syntactically before the part directive in F including the file that contains B in source order, then A is before B.
    • Otherwise B is before A.

Then B is after A if and only if A is before B.

In short, if A is above B, then A is before B. Otherwise, they are in sibling part subtrees and the directive in the subtree whose part directive occurs first is before the other.

This order is total. It effectively orders declarations by a pre-order depth-first traversal of the file tree, visiting declarations of a file in source order, and then recursing on part-directives in source order.

Declaration context

The context of a top-level declaration in a Dart file is the library of the associated tree of Dart files. The context of a member declaration in a type declaration named N is the set of type declarations (introductory or augmenting) named N in the enclosing set of Dart files.

In Dart without this feature, a declaration generally introduces an entity (a class, a method, a variable, etc). With the augmentation feature, such entities are introduced by a sequence of declarations rather than a single declaration. A single declaration can still do it, that's just a special case, but we need to talk about these sequences of declarations as being a single thing. The notion of a context helps us doing this by indicating the location where we need to look in order to find all those declarations. Note that this location does not have to be contiguous, it can consist of a set of ranges (e.g., the context of a member declaration can be a set of class declarations).

Some declarations do not match any of these cases (e.g., a local variable declaration in a method body), but this does not matter: We never need to talk about the context of a local variable.

Scoping

The static and instance member namespaces for a type or extension declaration, augmenting or not, are lexical only. Only the declarations (augmenting or not) declared inside the actual declaration are part of the lexical scope that member declarations are resolved in.

This means that a static or instance member declared in the augmented declaration of a class is not lexically in scope in a corresponding augmenting declaration of that class, just as an inherited instance member is not in the lexical scope of a class declaration.

If a member declaration needs to reference a static or instance member declared in another introductory or augmenting declaration of the same type, it can use this.name for instance members an TypeName.name for static members to be absolutely sure. Or it can rely on the default if name is not in the lexical scope at all, in which case it’s interpreted as this.name if it occurs inside a scope where a this is available. This approach is always potentially dangerous, since any third-party import adding a declaration with the same name would break the code. In practice that’s almost never a problem, because instance members and top-level declarations usually use different naming strategies.

Example:

// Main library "some_lib.dart":
import 'other_lib.dart';

part 'some_augment.dart';

const b = 37;

class C {
  static const int b = 42;
  bool isEven(int n) {
    if (n == 0) return true;
    return !_isOdd(n - 1);
  }
}

// Augmentation "some_augment.dart":
part of 'some_lib.dart';

import 'also_lib.dart';

augment class C {
  bool _isOdd(int n) => !this.isEven(n - 1);
  void printB() { print(b); }  // Prints 37
}

This code is fine. Code in C.isEven can refer to members added in the augmentation like _isOdd() because there is no other _isOdd in scope, and code in C._isOdd works too by explicitly using this.isEvent to ensure it calls the correct method.

You can visualize the namespace nesting sort of like this:

some_lib.dart       :
                    :<part of----------
.---------------------------------------.
| import scope:                         |
| other_lib imports                     |
'---------------------------------------'
         ^          :         ^
         |          :         |
         |          : .-----------------.
         |          : | import scope:   |
         |          : | also_lib imports|
         |          : '-----------------'
         |          :         |
.--------------------------------------.
| top-level declaration scope:         |
| const b = 37                         |
| class C (fully augmented class)      |
|                                      |
'--------------------------------------'
         ^          :         ^
         |          :         |
.-----------------. : .----------------.
| class C         | : | augment class C|
| const b = 42    | : | _isOdd()       |
| isEven()        | : |                |
'-----------------' : '----------------'
         ^          |         ^
         |          |         |
.-----------------. | .-----------------.
| C.isEven() body | | | C._isOdd() body |
'-----------------' | '-----------------'

Each part files has its own combined import scope, extending that of its parent, and its own member declarations scopes for each declared member, introducing a lexical scope for the declaration’s contents. In the middle, each passes through the shared library declaration namespaces for the top-level instances themselves.

It is a compile time error for both a static and instance member of the same name to be defined on the same type, even if they live in different lexical scopes. You cannot work around this restriction by moving the static member out to an augmentation, even though it would result in an unambiguous resolution for references to those members.

Type annotation inheritance

An augmenting declaration may have no type annotations for a return type, variable type, parameter type, or type parameter bound. In the last case, that includes omitting the extends keyword. For a variable or parameter, a var keyword may replace the type.

When applying an augmenting declaration that contains a type annotation at one of these positions, to a definition to be augmented, it's a compile-time error if the type denoted by the augmenting declaration is not the same type as the type that the augmented definition has at the corresponding position. An augmenting declaration can omit type annotations, but if it doesn't, it must repeat the type from the augmented definition.

Applying augmentations

An augmentation declaration D is a declaration marked with the new built-in identifier augment, which makes D augment a declaration D1 with the same name and in the same context as D. D1 is determined as being before D and after every other declaration with the same name and in the same context which is before D (that is, D1 is the greatest declaration which is smaller than D, according to the 'after' ordering). A compile-time error occurs if no declaration satisfies the requirements on D1.

We say that D1 is the declaration which is augmented by D.

Note that D1 can be an augmentation declaration or an introductory declaration.

An augmentation declaration does not introduce a new name into the surrounding scope. We could say that it attaches itself to the existing name.

Making augment a built-in identifier is language versioned, to make it non-breaking for pre-feature code.

Application order

The same declaration can be augmented multiple times by separate augmentation declarations. This occurs in the situation where an augmentation declaration has an augmented declaration which is itself an augmentation declaration, and so on, until an introductory declaration is reached. This will happen in a finite number of steps because of the nature of the before/after ordering.

Declarations that contribute to the same effective declaration, one introductory declaration and zero or more augmentation declarations with the same name and in the same context, are totally ordered by the after relation, with the introductory declaration being least, and the augmentation declarations greater than that.

In particular, if all the augmentation declarations occur on the same path in the tree of Dart files that constitute the current library then they are ordered by their depth in the tree.

This applies both to top-level declarations and to member declarations of, for example, class declarations.

Augmenting class-like declarations

A class, enum, extension, extension type, mixin, or mixin class declaration can be marked with an augment modifier:

augment class SomeClass {
  // ...
}

This means that instead of creating a new declaration, the augmentation modifies a corresponding declaration in the library (which is 'before' this one).

A class, enum, extension type, mixin, or mixin class augmentation may specify extends, implements and with clauses (when generally supported). The types in these clauses are appended to the introductory declarations’ clauses of the same kind, and if that clause did not exist previously, then it is added with the new types. All regular rules apply after this appending process, so you cannot have multiple extends on a class, or an on clause on an enum, etc.

Instance or static members defined in the body of the augmenting type, including enum values, are added to the instance or static namespace of the corresponding type in the augmented library. In other words, the augmentation can add new members to an existing type.

Instance and static members inside a class-like declaration may themselves be augmentations. In that case, they augment the corresponding members in the same context (one introductory declaration and zero or more augmenting declarations, all occurring before the current augmenting type declaration), according to the rules in the following subsections.

It's a compile-time error if a library contains two top-level declarations with the same name, and:

  • Neither is an augmenting declaration, or
  • one of the declarations is a class-like declarations and the other is not of the same kind, meaning that at either one is a class, mixin, enum, extension or extension type declaration, and the other is not the same kind of declaration.

It is a compile-time error if:

  • The augmenting declaration and augmented declaration do not have all the same modifiers: abstract, base, final, interface, sealed and mixin for class declarations, and base for mixin declarations.

    This is not a technical requirement, but it ensures that looking at either declaration shows the complete capabilities of the declaration. It also deliberately prevents an augmentation from introducing a restriction that isn't visible to a reader of the main declaration.

  • The augmenting declaration declares an extends clause for a class declaration, but one was already present (or the class was a mixin class declaration, which does not allow extends clauses).

  • An augmenting extension declares an on clause (this is a syntax error). We also do not allow adding further restrictions to a mixin declaration, so no further types can be added to its on clause, if it even has one. These restrictions could both be lifted later if we have a compelling use case, as there is no fundamental reason it cannot be allowed.

  • The type parameters of the augmenting declaration do not match the augmented declarations's type parameters. This means there must be the same number of type parameters with the exact same type parameter names (same identifiers) and bounds if any (same types, even if they may not be written exactly the same in case one of the declarations needs to refer to a type using an import prefix).

    Since repeating the type parameters is, by definition, redundant, this restriction doesn't accomplish anything semantically. It ensures that anyone reading the augmenting type can see the declarations of any type parameters that it uses in its body and avoids potential confusion with other top-level variables that might be in scope in the library augmentation.

Augmenting functions

A top-level function, static method, instance method, or operator may be augmented to replace or wrap the augmented body in additional code:

// Wrap the augmented function in profiling:
augment int slowCalculation(int a, int b) {
  var watch = Stopwatch()..start();
  var result = augmented(a, b);
  print(watch.elapsedMilliseconds);
  return result;
}

The augmentation replaces the augmented function’s body with the augmenting function’s body.

Inside the augmenting function’s body, a special augmented(…) expression may be used to execute the augmented function body. That expression takes an argument list matching the augmented function's parameter list, and it has the same return type as the enclosing function.

The augmenting function does not have to pass the same arguments to augmented(…) as were passed to it. It may invoke augmented once, more than once, or not at all.

An augmenting function declaration may have an empty body (;) in order to only augment the metadata or doc comments of the function. In this case the body of the augmented member is not altered.

It is a compile-time error if:

  • The function signature of the augmenting function does not exactly match the function signature of the augmented function. This means that any provided return types must be the same type; there must be same number or required and optional positional parameters, all with the same types (when provided), the same number of named parameters, each pairwise with the same name, same type (when provided) and same required and covariant modifiers, and any type parameters and their bounds (when provided) must be the same (like for type declarations).

    Since repeating the signature is, by definition, redundant, this doesn't accomplish anything semantically. But it ensures that anyone reading the augmenting function can see the declarations of any parameters that it uses in its body.

  • The augmenting function specifies any default values. Default values are defined solely by the introductory function.

  • An augmenting declaration uses augmented when the augmented declaration has no concrete implementation. Note that all external declarations are assumed to have an implementation provided by another external source, and they will throw a runtime exception when called if not.

Augmenting variables, getters, and setters

While the language treats variables, getters, and setters as mostly interchangeable, within augmentations we do not allow augmenting getters and setters with variables. Since augmentations are tightly coupled to the libraries they augment, this restriction has minimal impact, and it does not greatly affect the ability of a library to change a field to a getter/setter pair or vice-versa.

You can think of variable, getter, and setter declarations all as ways to define a higher-level "property" construct. A property has a name and a type. It may have one or more other capabilities:

  • A backing storage location. You get this when you declare a variable which is not abstract and not external. Having a storage location enables (and often requires) having the variable initialized by generative constructors. A variable may also have an initializing expression that gets run either lazily for top-level and static variables, or at object construction/initialization time for instance variables.

  • A getter function. This function’s body is provided explicitly when you declare a getter. A variable declaration provides an implicit getter body that returns the value in the backing storage location. (Late variables do some additional checking in that implicit body.)

  • A setter function. A setter declaration provides a body explicitly. A non-final variable declaration provides an implicit setter body that stores the given value in the storage location. (Again, late variables do some additional updates and/or checks.)

Variable declarations may be marked abstract or external and, if so, those are mapped over to the corresponding getter and setter functions.

An abstract variable declaration is equivalent to an abstract getter declaration, and if not final, also an abstract setter declaration. An external variable defines an external getter and, if not final, an external setter. Unlike abstract declarations, they are considered to have a concrete implementation.

Variables which require an initializer expression (such as those which have a non-nullable type and are not marked late) need not initially be defined with one, as long as there exists some augmentation which supplies it.

Augmentations on variables, getters, and setters works mostly at the level of these separate capabilities. For example, augmenting a variable with a getter replaces the augmented variable's implicit getter body with the augmenting getter's.

More specifically:

  • Augmenting with a getter: An augmenting getter can augment a getter declaration, or the implicit getter of a variable declaration, with all prior augmentations applied, by replacing the body of the augmented getter with the body of the augmenting getter. Inside the augmenting getter’s body, an augmented expression executes the augmented getter’s body.

    An augmenting getter declaration may have an empty body (;) in order to only augment the metadata or doc comments of the getter. In this case the body of the augmented getter is not altered.

    Synthetic getters cannot be augmented with metadata or doc comments.

  • Augmenting with a setter: An augmenting setter can augment a setter declaration, or the implicit setter of a variable declaration, with all prior augmentations applied, by replacing the augmented setter’s body with the augmenting setter’s body. Inside the augmenting setter’s body, an augmented = <expression> assignment invokes the augmented setter with the value of the expression.

    An augmenting setter declaration may have an empty body (;) in order to only augment the metadata or doc comments of the setter. In this case the body of the augmented setter is not altered.

    Synthetic setters cannot be augmented with metadata or doc comments.

  • Augmenting a getter and/or setter with a variable: This is a compile-time error in all cases. Augmenting an abstract or external variable with a variable is also a compile-time error, as those are actually just syntax sugar for a getter and possibly a setter. An augmenting variable replaces its augmented variable’s initializer expression, and that can only be done on a declaration that can have an initializer expression.

    We may decide in the future to allow augmenting abstract getters, setters, or variables with variables, but for now you can instead use the following workaround:

    • Add a new field.
    • Augment the getter and/or setter to delegate to that field.

    If a non-abstract, non-external variable is augmented by an augmenting getter or setter, you can still augment the variable, as you are only augmenting the initializer, metadata, or doc comments of the augmented variable. This is not considered to be augmenting the augmenting getter or setter, since those are not actually altered.

    The reason for this compile time error is that whether a member declaration is a field versus a getter/setter is a visible property of the declaration inside the same class or even library:

    • It determines whether the member can be initialized in a constructor initializer list.
    • It is also a visible distinction when introspecting on a program with the analyzer, macros, or mirrors.

    When a declaration is augmented, we don't want the augmentation to be able to change any of the known properties of the existing member being augmented. For example, we don't allow you to augment a method with a getter that returns a function. Augmenting a getter/setter pair with a field would change the "can be used in a constructor initializer" property, so we forbid it. Augmenting a field with a getter/setter doesn't change that property so it is allowed.

  • Augmenting a variable with a variable: Augmenting a variable with a variable only alters its initializer, metadata, or doc comments. As usual, external and abstract variables cannot augment their initializing expression, since it does not exist.

    Augmenting initializer expressions replace the augmented initializer (or provide one where none existed previously). The augmenting initializer may use an augmented expression which executes the augmented initializer expression (if present) when evaluated. If no initializer is provided then the augmented initializer is not altered.

    The late property of a variable must always be consistent between the augmented variable and its augmenting variables.

    If the introductory variable declaration does not have a type annotation, then the variable's declared type is found using only that declaration, without looking at any further augmenting declarations. The type can either be inferred from an initializer expression of the introductory variable declaration, be inherited from a superinterface for an instance variable, or default to a type of dynamic if neither applies. This ensures that augmenting a variable doesn't change its type. That is necessary to ensure that macros cannot change the signature of a declaration, a signature which may have been depended on by other code, or other macros.

It is a compile-time error if:

  • The introductory and augmenting declarations do not have the same declared types (return type for getters, parameter type for setters, declared type for variables). This only applies where types are not omitted in the augmenting declaration.

  • An augmenting declaration uses augmented when the augmented declaration has no concrete implementation. Note that all external declarations are assumed to have an implementation provided by another external source, and otherwise they will throw a runtime error when called.

  • An augmenting variable’s initializing expression uses augmented, and the stack of augmented declarations do not include a variable with an explicit initializing expression. For nullable fields, the implicit null initialization only happens if there is no explicit initializer after the entire stack of augmentations has been applied.

  • A non-writable variable declaration is augmented with a setter. (Instead, the author can declare a non-augmenting setter that goes alongside the implicit getter defined by the final variable.) A non-writable variable declaration is any that does not introduce a setter, including non-late final variables, late final variables with an initializer, and const variables.

  • A non-final variable is augmented with a final variable. We don't want to leave the augmented setter in a weird state.

    • A final variable can be augmented with a non-final augmenting variable, and that will not add any setter. An augmenting variable declaration only affects the initializer expression, not setters.

  • A variable is augmented with another variable, and one is late and the other is not. (Augmentation cannot change late-ness, and since being late does affect the initializer expression, the augmenting variable is required to repeat the late.)

  • A getter or setter declaration is augmented by an augmenting variable.

  • A late final variable with no initializer expression is augmented by an augmenting variable with an initializer expression. A late final variable with no initializer has a setter, while one with an initializer does not. An augmentation must not change whether there is a setter.

  • A const variable is augmented by an augmenting getter. (TODO: Can a const variable be augmented by another const variable, changing its value, or is that too weird?)

  • An abstract variable is augmented with a non-abstract variable.

  • An external variable is augmented with an abstract variable.

Augmenting enum members

Some enum members can not be augmented: It is a compile-time error if an augmenting declaration in an enum declaration (introductory or augmenting) has the name values, index, hashCode, or ==.

It has always been an error for an enum declaration to declare a member named index, hashCode, ==, or values, and this rule just clarifies that this error is applicable for augmenting declarations as well.

Enum values can only be augmented by enum values, and the implicit getter introduced by them is not augmentable. The only thing you are allowed to do when augmenting an enum value is add metadata annotations or doc comments.

When augmenting an enum value, no constructor invocation should be provided. The original value is always used, and the explicit constructor invocation (if present) should not be copied.

New enum values may be defined in an augmenting enum, and they will be appended to the current values of the declaration in augmentation application order.

Augmenting an existing enum value never changes the order in which it appears in values.

For example:

// main.dart
part 'a.dart';
part 'c.dart';

enum A {
  first,
  second.custom(1);

  final int b;

  const A() : b = 0;

  const A.custom(this.b);
}
}

// a.dart
part of 'main.dart';
part 'b.dart';

augment enum A {
  third;

  /// Some doc comment
  augment first; // This is still `first` in values.

  @someAnnotation
  augment second; // Don't repeat the argument list, original is used.
}

// b.dart
part of 'a.dart';

augment enum A {
  fourth;
}

// c.dart
part of 'main.dart';

augment enum A {
  fifth;

  // Error, enum value augmentations cannot have an explicit constructor
  // invocation.
  augment third.custom(3);
}

Then A.values is [A.first, A.second, A.third, A.fourth, A.fifth].

It is a compile-time error if:

  • An augmenting getter is defined for an enum value. An enum value counts as a constant variable.
  • An enum value augmentation provides an explicit constructor invocation.

Augmenting constructors

Constructors are (as always) more complex. We have many kinds of constructors, and what it means to augment each is different. For the purposes of this section we will call out three specific kinds of constructors:

Non-redirecting generative constructors: These always produce a new instance, and have an optional initializer list and optional body. Non-redirecting factory constructors: These are much like static methods, and might return a subtype of the current type. They also may not create a new instance, but return an existing one. They must have a body. Redirecting constructors: Both generative and factory constructors can be redirecting, although the syntax looks slightly different for each.

It may not always be apparent whether a constructor is redirecting or not based on a given declaration (if there is no body, initializer list, or redirecting constructor invocation). These constructors are considered to be "potentially redirecting" or "potentially non-redirecting". An augmentation may alter this property by augmenting a constructor in a way that makes it concretely redirecting or not.

It is a compile-time error if:

  • The signature of the constructor augmentation does not match the original constructor. It must have the same number of positional parameters, the same named parameters, and matching parameters must have the same type, optionality, and any required modifiers must match. Any initializing formals and super parameters must also be the same in both constructors.

  • The augmenting constructor parameters specify any default values. Default values are defined solely by the introductory constructor.

  • The introductory constructor is const and the augmenting constructor is not or vice versa.

  • The introductory constructor is marked factory and the augmenting constructor is not, or vice versa.

  • The introductory constructor has a super initializer (super constructor invocation at the end of the initializer list) and the augmenting constructor does too. An augmentation can replace the implicit default super() with a concrete super-invocation, but cannot replace a declared super constructor. (TODO: Why not? We allow "replacing implementation", and this is something like that.)

  • The resulting constructor is not valid (it has a redirection as well as some initializer list elements, or it has multiple super initializers, etc).

  • A non-redirecting constructor augments a constructor which is not potentially non-redirecting.

  • A redirecting constructor augments a constructor which is not potentially redirecting.

Non-redirecting generative constructors

These are probably the most complex constructors, but also the most common.

At a high level, a non-redirecting generative constructor marked augment may:

  • Augment the constructor with an additional constructor body (bodies are invoked in augmentation order, starting at the introductory declaration).

  • Add initializers (and/or asserts) to the initializer list, as well as a super call at the end of the initializer list.

Non-redirecting factory constructors

A non-redirecting factory constructor marked augment works in the same way as a normal function augmentation.

If it has a body, it replaces the body of the augmented constructor (if present), and it may invoke the augmented body by calling augmented(arguments).

Redirecting generative constructors

A redirecting generative constructor marked augment adds its redirection to the augmented constructor.

This converts it into a redirecting generative constructor, removing the potentially non-redirecting property of the constructor.

It is a compile-time error if:

  • The augmented constructor has any initializers.
  • The augmented constructor has a body.
  • The augmented constructor has a redirection.

This redirecting generative constructor now behaves exactly like any other redirecting generative constructor when it is invoked.

Redirecting factory constructors

A redirecting factory constructor marked augment adds its factory redirection (e.g., = C<int>.name) to the augmented constructor.

The result of applying the augmenting constructor is a redirecting factory constructor with the same target constructor designation as the augmenting constructor. This removes the potentially non-redirecting property of the constructor.

It is a compile-time error if:

  • The augmented factory constructor has a body, or it is redirecting.

Augmenting extension types

When augmenting an extension type declaration, the parenthesized clause where the representation type is specified is treated as a constructor that has a single positional parameter, a single initializer from the parameter to the representation field, and an empty body. The representation field clause must be present on the declaration which introduces the extension type, and must be omitted from all augmentations of the extension type.

This means that an augmentation can add a body to an extension type's implicit constructor, which isn't otherwise possible. This is done by augmenting the constructor in the body of the extension type. Note that there is no guarantee that any instance of an extension type will have necessarily executed that body, since you can get instances of extension types through casts or other conversions that sidestep the constructor. For example:

extension type A(int b) {
  augment A(int b) {
    assert(b > 0);
  }
}

This is designed in anticipation of supporting primary constructors on other types in which case the extension type syntax will then be understood by users to be a primary constructor for the extension type.

The extension type's representation object is not a variable, even though it looks and behaves much like one, and it cannot be augmented as such. It is a compile time error to have any augmenting declaration with the same name as the representation object.

It is a compile time error if:

  • An extension type augmentation contains a representation field clause.

Augmenting external members

When augmenting an external member, it is assumed that a real implementation of that member has already been filled by some tool prior to any augmentations being applied. Thus, it is allowed to use augmented from augmenting members on external declarations, but it may throw a NoSuchMethodError error at runtime if no implementation was in fact provided.

NOTE: Macros should not be able to statically tell if an external body has been filled in by a compiler, because it could lead to a different result on different platforms or tools.

TODO: Should we add a syntax to let the augmentation dynamically detect whether there is an external implementation to call?

Augmenting with metadata annotations and doc comments

All declarations can be augmented with metadata annotations and/or doc comments directly preceding an augmenting declaration.

In both cases, these should be appended to existing metadata or doc comments. For metadata annotations, these may trigger additional macro applications.

Augmented expression

The exact result of an augmented expression depends on what is being augmented, but it generally follows the same rules as any normal identifier:

  • Augmenting getters: Within an augmenting getter augmented invokes the augmented getter and evaluates to its return value. If augmenting a variable with a getter, this will invoke the implicitly induced getter from the augmented variable declaration.

  • Augmenting setters: Within an augmenting setter augmented must be followed by an = and will directly invoke the augmented setter. If augmenting a variable with a setter, this will invoke the implicitly induced setter from the augmented variable declaration.

  • Augmenting fields: Within an augmenting variable declaration, augmented can only be used in an initializer expression, and refers to the augmented variable's initializing expression, which is immediately evaluated.

    It is a compile-time error to use augmented in an augmenting variable's initializer if the member being augmented is not a variable declaration with an initializing expression.

  • Augmenting functions: Inside an augmenting function body (including factory constructors but not generative constructors) augmented refers to the augmented function. Tear-offs are not allowed, and this function must immediately be invoked.

  • Augmenting non-redirecting generative constructors: Unlike other functions, augmented has no special meaning in non-redirecting generative constructors. It is still a reserved word inside the body of these constructors, since they are within the scope of an augmenting declaration.

    There is instead an implicit order in which these augmented constructors are invoked, and they all receive the same arguments. See this section for more information.

  • Augmenting operators: When augmenting an operator, augmented refers to the augmented operator method, which must be immediately invoked using function call syntax. For example, when augmenting operator + you could use augmented(1) to call the augmented operator, and when augmenting operator []= you would use the augmented(key, value) syntax.

    • Note that augmented in such an augmenting operator method body is not an expression by itself, and cannot be used to tear off the augmented operator method. Similar to super, it is a syntactic form which can only be used in limited ways.

  • Augmenting enum values: When augmenting an enum value, augmented has no meaning and is not allowed.

In all relevant cases, if the augmented member is an instance member, it is invoked with the same value for this.

Assume that the identifier augmented occurs in a source location where no enclosing declaration is augmenting. In this case, the identifier is taken to be a reference to a declaration which is in scope.

In other words, augmented is just a normal identifier when it occurs anywhere other than inside an augmenting declaration.

Note that, for example, augmented() is an invocation of the augmented function or method when it occurs in an augmenting function or method declaration. (In the latter case, the augmenting method declaration must occur inside an augmenting type-introducing declaration, e.g., an augmenting class or mixin declaration). This is also true if augmented() occurs inside a local function declaration inside the body of that function or method declaration. We could say that augmented is a contextual reserved word because it is unable to refer to a declaration in scope when it occurs inside an augmenting declaration, it always has the special meaning which is associated with augmentations.

A compile-time error occurs if a declaration with the basename augmented occurs in a location where any enclosing declaration is augmenting. This error is applicable to all such declarations, e.g., local functions, local variables, parameters, and type parameters.

Consider a non-augmenting member declaration Dm that occurs inside an augmenting type declaration Dt. A compile-time error occurs if the identifier augmented occurs in Dm.

For example, inside augment class C we could have a declaration like void f() {...augmented()...}. This is an error because the outer augment forces the meaning of augmented to be about augmentation in the entire scope, but the method declaration is introductory and hence there is no earlier declaration to augment.

Dynamic semantics

The application of augmentation declarations to an augmented declaration produces something that looks and behaves like a single declaration: It has a single name, a single type or function signature, and it’s what all references to the name refers to inside and outside of the library.

Unlike before, that single semantic declaration now consists of multiple syntactic declarations (one introductory declaration, the rest augmenting declarations, with a given augmentation application order), and the properties of the combined semantic declaration can be derived from the syntactic declarations.

We redefine a number of semantic functions to now work on a stack of declarations (the declarations for a name in bottom to top order), so that existing semantic definitions keep working.

Example: Class declarations

Super-declarations

The specification of class modifiers introduced a number of predicates on declarations, to check whether the type hierarchy is well formed and the class modifiers are as required, before the static semantics have even introduced types yet. We modify those predicates to apply to a stack of augmenting declarations and an introductory declaration as follows:

  • A a non-empty stack of syntactic class declarations, C, has a declaration D as declared super-class if:
    • C starts with an (augmenting or not) class declaration C0 and either
      • C0 has an extends clause whose type clause denotes the declaration D, or
      • C0 is an augmenting declaration, so C continues with a non-empty Crest, and Crest has D as declared super-class.
  • A a non-empty stack of syntactic class declarations, C, has a declaration D as declared super-interface if:
    • C starts with an (augmenting or not) class declaration C0 and either
      • C0 has an implements clause with an entry whose type clause denotes the declaration D, or
      • C0 is an augmenting declaration, so C continues with a non-empty Crest, and Crest has D as declared super-interface.
  • A a non-empty stack of syntactic class declarations, C, has a declaration D as declared super-mixin if:
    • C starts with an (augmenting or not) class declaration C0 and either
      • C0 has a with clause with an entry whose type clause denotes the declaration D, or
      • C0 is an augmenting declaration, so C continues with a non-empty Crest, and Crest has D as declared super-mixin.

Members

A class declaration stack, C, of a one non-augmenting and zero or more augmenting class declarations, defines an augmented interface (member signatures) and augmented implementation (instance members declarations) based on the individual syntactic declarations.

A non-empty class declaration stack, C, has the following set of instance member declarations:

  • Let Ctop be the latest declaration of the stack, and Crest the rest of the stack.
  • If Ctop is a non-augmenting declaration, the declarations of C is the set of syntactic instance member declarations of Ctop.
  • Otherwise let P be the set of member declarations of the non-empty stack Crest.
  • and the member declarations of C is the set R defined as containing only the following elements:
    • A singleton stack of each syntactic instance member declaration M of Ctop, where M is a non-augmenting declaration.
    • The elements N of P where Ctop does not contain an augmenting instance member declaration with the same name (mutable variable declarations have both a setter and a getter name).
    • The stacks of a declaration M on top of the stack N, where N is a member of P, M is an augmenting instance member declaration of Ctop, and M has the same name as N.

And we can whether such an instance member declaration stack, C, defines an abstract method as:

  • Let Ctop be the latest element of the stack and Crest the rest of the stack.
  • If Ctop is a non-variable declaration, and is not declared abstract, the C doe
  • If Ctop declares a function body, then C does not define an abstract method.
  • Otherwise C defines an abstract method if Crest defines an abstract method.

(This is just for methods, we will define it more generally for members, including variable declarations.)

Example: Instance methods

Properties

Similarly we can define the properties of stacks of member declarations.

For example, we define the augmented parameter list of a non-empty stack, C, of augmentations on an introductory function declaration as:

  • Let Ctop be the latest element of the stack and Crest the rest of the stack.
  • If Ctop is not an augmenting declaration, its augmented parameter list is its actual parameter list. (And Crest is known to be empty.)
  • Otherwise Ctop is an augmenting declaration with a parameter list which must have the same parameters (names, positions, optionality and types) as its augmented declaration, except that it is not allowed to declare default values for optional parameters.
    • Let P be the augmented parameter list of Crest.
    • The augmented parameter list of Ctop is then the parameter list of Ctop, updated by adding to each optional parameter the default value of the corresponding parameter in P, if any.

This will usually be exactly the parameter list of the introductory declaration, but the ordering of named parameters may differ. This is mostly intended as an example, in practice the augmented parameter list can just be the parameter list of the introductory declaration, but it’s more direct and clearly correct to use the actual parameter list of the declaration when creating the parameter scope that its body will run in.

Similarly we define the augmented function type of the declaration stack. Because of the restrictions we place on augmentations, they will all have the same function type as the introductory declaration, but again it’s simpler to assign a function type to every declaration.

Invocation

When invoking an instance member on an object, the current specification looks up the corresponding implementation on the class of the runtime-type of the receiver, traversing super-classes, until it it finds a non-abstract declaration or needs to search past Object. The specification then defines how to invoke that method declaration, with suitable contexts and bindings.

We still define the same thing, only the result of lookup is not a single declaration, but a stack of augmenting declarations on top of an introductory declaration, and while searching, we skip past declaration stacks that define an abstract method. The resulting stack is the member definition, or semantic declaration, which is derived from the syntactic declarations in the source.

Invoking a stack, C, of instance method declarations on a receiver object o with an argument list A and type arguments T, is then defined as follows:

  • Let Ctop be the latest declaration on the stack (the last applied augmentation in augmentation application order), and Crest the rest of the stack.
  • If Ctop has a function body B then:
    • Bind actuals to formals (using the usual definition of that), binding the argument list A and type arguments T to the augmented parameter list of Ctop and type parameters of Ctop. This creates a runtime parameter scope which has the runtime class scope as parent scope (the lexical scope of the class, except that type parameters of the class are bound to the runtime type arguments of those parameters for the instance o).
    • Execute the body B in this parameter scope, with this bound to o.
    • If B contains an expression of the form augmented<TypeArgs>(args) (type arguments omitted if empty), then:
      • The static type of augmented is the augmented function type of Crest. The expression is type-inferred as a function value invocation of a function with that static type.
      • To evaluate the expression, evaluate args to an argument list A2, invoke Crest with argument list A2 and type arguments that are the types of TypeArgs. The result of augmented<TypeArgs>(args) is the same as the result of that invocation (returned value or thrown error).
    • There would have been a compile-time error if there is no earlier declaration with a body.
    • The result of invoking C is the returned or thrown result of executing B.
  • Otherwise, the result of the invocation of C is the result of invoke Crest on o with argument list A and type arguments T.
    • This will eventually find a body to execute, otherwise C would have defined an abstract method, and would not have been invoked to begin with.

Tooling

Documentation comments

Documentation comments are allowed in all the standard places in library augmentations. It is up to the tooling to decide how to present such documentation comments to the user, but they should generally be considered to be additive, and should not completely override the original comment. In other words, it is not the expectation that augmentations should duplicate the original documentation comments, but instead provide comments that are specific to the augmentation.

Path requirement lint suggestion

One issue with the augmentation application order is that it is not stable under reordering of part directives. Sorting part directives can change the order that augmentation applications in separate included sub-trees are applied in.

To help avoiding issues, we want to introduce a lint which warns if a library is susceptible to part file reordering changing augmentation application order. A possible name could be augmentation_ordering.

Its effect would be to report a warning if for any two (top-level) augmenting declarations with name n, one is not above the other.

The lint would only apply to user-written augmenting declarations, it should not include macro generated augmentations. Those are placed where the macro processor chooses to place them, usually after all other augmentations.

If the lint is satisfied, then all augmenting declarations are ordered by the before relation, which means that no two of them can be in different sibling parts of the same file, and therefore all the augmenting declarations occur along a single path down the part-file tree. This ensures that part file directive ordering has no effect on augmentation application order.

The language specification doesn’t specify lints or warnings, so this lint suggestion is not normative. We wish to have the lint, and preferably include it in the "recommended" lint set, because it can help users avoid accidental problems. We want it as a lint instead of a language restriction so that it doesn’t interfere with macro-generated code, and so that users can // ignore: it if they know what they’re doing.

Changelog

1.35

  • Reorganize sections.

1.34

  • Revert some errors introduced in version 1.28.

    • An abstract variable can now be augmented with non-abstract getters and setters.
    • External variables can now be augmented with abstract getters and setters.

1.33

  • Change the grammar to remove the primary constructor parts of an augmenting extension type declaration.

1.32

  • Specify that variables which require an initializer can have it defined in any augmentation.
  • Specify that the implicit null initialization is not applied until after augmentation.

1.31

  • Specify that it is an error to have a static and instance member with the same name in the fully merged declaration.

1.30

  • Simplify extension type augmentations, don't allow them to contain the representation type at all.

1.29

  • Simplify enum value augmentations, no longer allow altering the constructor invocation.

1.28

  • Explicitly disallow augmenting abstract variables with non-abstract variables, getters, or setters.
  • Explicitly disallow augmenting external declarations with abstract declarations.
  • Remove error when augmenting an abstract or external variable with a variable (allowed for adding comments/annotations).

1.27

  • Specify that representation objects for extension types cannot be augmented.

1.26

  • Recreate the change made in 1.23 (which was undone by accident).

1.25

  • Clarify that augmentations can occur in the same type-introducing declaration body, even in a non-augmenting declaration.
  • Update some occurrences of old terminology with new terms.

1.24

  • Allow augmentations which only alter the metadata and/or doc comments on various types, and specify behavior.

1.23

  • Change augmented operator invocation syntax to be function call syntax.

1.22

  • Unify augmentation libraries and parts. Parts with imports specification moved into separate document, as a stand-alone feature that is not linked to augmentations.
  • Augmentation declarations can occur in any file, whether a library or part file. Must occur "below" the introductory declaration (later in same file or sub-part) and "after" any prior applied augmentation that it modifies (below, or in a later sub-part of a shared ancestor).
  • Suggest a stronger ordering lint, where the augmentation must be "below" the augmentation it is applied after. That imples that all declarations with the same name are on the same path in the library file tree, so that reordering part directives does not change augmentation application order.
  • Change the lexical scope of augmenting class-like declarations to only contain the member declarations that are syntactically inside the same declaration, rather than collecting all member declarations from all augmenting or non-augmenting declarations with the same name, and making them all available in each declaration.
  • Avoid defining a syntactic merging, since it requires very careful scope management, which isn’t necessary if we can just extend properties that are currently defined for single declarations to the combination of a declaration plus zero or more augmentations.

1.21

  • Add a compile-time errors for wrong usages of augmented.

1.20

  • Change the extensionDeclaration grammar rule such that an augmenting extension declaration cannot have an on clause. Adjust other rules accordingly.

1.19

  • Change the phrase 'augmentation library' to 'library augmentation', to be consistent with the rename which was done in 1.15.

1.18

  • Add a grammar rule for enumEntry, thus allowing them to have the keyword augment.

1.17

  • Introduce compile-time errors about wrong structures in the graph of libraries and augmentation libraries formed by directives like import and import augment (#3646).

1.16

  • Update grammar rules and add support for augmented type declarations of all kinds (class, mixin, extension, extension type, enum, typedef).

  • Specify augmenting extension types. Clarify that primary constructors (which currently only exist for extension types) can be augmented like other constructors (#3177).

1.15

  • Change library augment to augment library.

1.14

  • Change augment super to augmented.

1.13

  • Clarify which clauses are (not) allowed in augmentations of certain declarations.
  • Allow adding an extends clause in augmentations.

1.12

  • Update the behavior for variable augmentations.

1.11

  • Alter and clarify the semantics around augmenting external declarations.
  • Allow non-abstract classes to have implicitly abstract members which are implemented in an augmentation.

1.10

  • Make augment a built-in identifier.

1.9

  • Specify that documentation comments are allowed, and should be considered to be additive and not a complete override of the original comment. The rest of the behavior is left up to implementations and not specified.

1.8

  • Specify that augmented libraries and their augmentations must have the same language version.

  • Specifically call out that augmentations can add and augment enum values, and specify how that works.

1.7

  • Specify that augmentations must contain all the same keywords as the original declaration (and no more).

1.6

  • Allow class augmentations to use different names for type parameters. This isn't particular valuable, but is consistent with functions augmentations which are allowed to change the names of positional parameters.

  • Specify that a non-augmenting declaration must occur before any augmentations of it, in merge order.

  • Specify that augmentations can't have parts (#2057).

1.5

  • Augmentation libraries share the same top-level declaration and private scope with the augmented library and its other augmentations.

  • Now that enums have members, allow them to be augmented.

  • Compile-time error if a non-late augmenting instance variable calls the initializer for a late one.

1.4

  • When inferring the type of a variable, only the original variable's initializer is used.

1.3

  • Constructor and function augmentations can't define default values.

1.2

  • Specify that augmenting constructor initializers are inserted before the original constructor's super or redirecting initializer if present (#2062).
  • Specify that an augmenting type must replicate the original type's type parameters (#2058).
  • Allow augmenting declarations to add metadata annotations and macro applications (#2061).

1.1

  • Make it an error to apply the same augmentation multiple times (#1957).
  • Clarify type parameters and parameter modifiers in function signature matching (#2059).

1.0

Initial version.