Skip to content
This repository was archived by the owner on Feb 22, 2023. It is now read-only.

[flutter_plugin_tool] Move branch-switching logic from tool_runner.sh to tool #4268

Merged
merged 10 commits into from
Aug 31, 2021

Conversation

stuartmorgan-g
Copy link
Contributor

@stuartmorgan-g stuartmorgan-g commented Aug 25, 2021

Eliminates the remaining logic from tool_runner.sh, completing the goal of migrating repository tooling off of bash (both to make maintenance easier, and to better support Windows both locally and in CI). Its branch-based logic is now part of the tool itself, via a new --packages-for-branch flag (which is hidden in help since it's only useful for CI).

Part of flutter/flutter#86113

Pre-launch Checklist

  • I read the Contributor Guide and followed the process outlined there for submitting PRs.
  • I read the Tree Hygiene wiki page, which explains my responsibilities.
  • I read and followed the relevant style guides and ran the auto-formatter. (Note that unlike the flutter/flutter repo, the flutter/plugins repo does use dart format.)
  • I signed the CLA.
  • The title of the PR starts with the name of the plugin surrounded by square brackets, e.g. [shared_preferences]
  • I listed at least one issue that this PR fixes in the description above.
  • I updated pubspec.yaml with an appropriate new version according to the pub versioning philosophy.
  • I updated CHANGELOG.md to add a description of the change.
  • I updated/added relevant documentation (doc comments with ///).
  • I added new tests to check the change I am making or feature I am adding, or Hixie said the PR is test exempt.
  • All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

@stuartmorgan-g stuartmorgan-g marked this pull request as draft August 25, 2021 20:37
@google-cla google-cla bot added the cla: yes label Aug 25, 2021
@stuartmorgan-g stuartmorgan-g force-pushed the tool-runner-replacement branch 2 times, most recently from 62613dd to 31ba457 Compare August 30, 2021 18:54
@stuartmorgan-g stuartmorgan-g changed the title [flutter_plugin_tool] Remove dependency on tool_runner.sh [flutter_plugin_tool] Move branch-switching logic from tool_runner.sh to tool Aug 30, 2021
@stuartmorgan-g stuartmorgan-g marked this pull request as ready for review August 30, 2021 19:41
Copy link
Member

@gaaclarke gaaclarke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just a handful of comments / nits.

.cirrus.yml Outdated
@@ -224,7 +234,7 @@ task:
### iOS+macOS tasks ***
- name: lint_darwin_plugins
script:
- ./script/tool_runner.sh podspecs
- dart $PLUGIN_TOOL podspecs $COMMON_TOOL_FLAGS
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is really an improvement because now we've introduced the risk of someone forgetting COMMON_TOOL_FLAGS. I looked at the cirrus documentation and it doesn't look like you could make a function like $RUN_PLUGIN_TOOL(podspecs).

If you made the command being run not have to be positional with the presence of a new flag you could extract RUN_PLUGIN_TOOL=dart $PLUGIN_TOOL $COMMON_TOOL_FLAGS --command

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is really an improvement because now we've introduced the risk of someone forgetting COMMON_TOOL_FLAGS.

Yes, I went back and forth on this for the same reason, and didn't see a good solution. The main benefit is that it eliminates the strange setup where tool_runner.sh is reading a magic env variable, and .cirrus.yml sets the magic variable, but it's totally non-obvious in each one in isolation what this variable is.

But I'll just revert this part for now and revisit it later.

If you made the command being run not have to be positional with the presence of a new flag you could extract RUN_PLUGIN_TOOL=dart $PLUGIN_TOOL $COMMON_TOOL_FLAGS --command

I looked into this a bit, and AFAICT allowing flags to be before the command with CommandRunner requires declaring them at different level that has totally different (and much more awkward) affordances for reading the flags. I can probably move just this one flag as a hack, but it seems ugly. Maybe I'll find a better solution when revisiting.

final GitVersionFinder gitVersionFinder = await retrieveVersionFinder();
final List<String> changedFiles =
await gitVersionFinder.getChangedFiles();
if (!_changesRequireFullTest(changedFiles)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit fyi: This would be more efficient as a Stream<String> instead of Future<List<String>>.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I can change it in a follow-up. Neither the API nor the way it's being used are new in this PR, I'm just rearranging the flow a bit.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, having looked at the details I'm not clear how it would be more efficient. getChangedFiles is taking the full stdout from git (the process API for the tools aren't stream-based) and splitting it by lines, giving a list. So returning a Stream instead of a List doesn't change anything there.

At this call site I need this list in two places: _changesRequireFullTest and _getChangedPackages. I can't consume the stream twice, so I would need to .toList() it here.

So the difference would be that instead of taking a List and returning it to a caller that needs a List, I would be taking a List, converting it to a Stream, and giving the stream to the caller to immediately turn back to a List. Am I missing a different way of structuring this?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, if you are going to cache the result it doesn't make a difference, it's more of an academic change.

Returning a Stream is better because it allows people to process the output while it is being parsed. This is especially helpful when you are working on input from files/pipes. This also means the operation isn't memory bound and if a consumer decides to stop consuming mid-stream resources won't be wasted processing the output that happens after it stops.

It doesn't mean a lick of difference if you cache the results though so you can process it twice.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Returning a Stream is better because it allows people to process the output while it is being parsed. This is especially helpful when you are working on input from files/pipes.

The supply-side issue here is that the git package doesn't steam git output, it uses run and returns the full output as a String.

final Set<String> packages = <String>{};
for (final String path in allChangedFiles) {
for (final String path in changedFiles) {
final List<String> pathComponents = path.split('/');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not introduced in this change, but is it a problem we aren't using pathSeparator here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In practice it's not because this is only ever called with git output, which always uses POSIX paths. I changed this to p.posix.split and added a comment to the method saying that they must be POSIX paths as a quick improvement. (The better, more involved solution would be to use Files rather than Strings.)

@stuartmorgan-g stuartmorgan-g merged commit ffe53ec into flutter:master Aug 31, 2021
@stuartmorgan-g stuartmorgan-g deleted the tool-runner-replacement branch August 31, 2021 14:54
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Aug 31, 2021
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Aug 31, 2021
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Aug 31, 2021
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Sep 1, 2021
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Sep 1, 2021
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Sep 7, 2021
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Sep 8, 2021
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Sep 8, 2021
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Sep 8, 2021
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Sep 8, 2021
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Sep 8, 2021
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Sep 9, 2021
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Sep 9, 2021
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Sep 9, 2021
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Sep 9, 2021
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Sep 9, 2021
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Sep 9, 2021
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Sep 9, 2021
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Sep 9, 2021
fotiDim pushed a commit to fotiDim/plugins that referenced this pull request Sep 13, 2021
… to tool (flutter#4268)

Eliminates the remaining logic from tool_runner.sh, completing the goal of migrating repository tooling off of bash (both to make maintenance easier, and to better support Windows both locally and in CI). Its branch-based logic is now part of the tool itself, via a new `--packages-for-branch` flag (which is hidden in help since it's only useful for CI).

Part of flutter/flutter#86113
amantoux pushed a commit to amantoux/plugins that referenced this pull request Sep 27, 2021
… to tool (flutter#4268)

Eliminates the remaining logic from tool_runner.sh, completing the goal of migrating repository tooling off of bash (both to make maintenance easier, and to better support Windows both locally and in CI). Its branch-based logic is now part of the tool itself, via a new `--packages-for-branch` flag (which is hidden in help since it's only useful for CI).

Part of flutter/flutter#86113
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants