Skip to content

flutter_tools\test\asset_bundle_test.dart is flaky on windows #34446

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
jonahwilliams opened this issue Jun 13, 2019 · 11 comments · Fixed by #42597
Closed

flutter_tools\test\asset_bundle_test.dart is flaky on windows #34446

jonahwilliams opened this issue Jun 13, 2019 · 11 comments · Fixed by #42597
Assignees
@jonahwilliams
Labels
c: flake Tests that sometimes, but not always, incorrectly pass

Comments

@jonahwilliams
Copy link
Member

test\asset_bundle_test.dart: AssetBundle.build handle removal of wildcard directories
@Hixie
Copy link
Contributor

Hixie commented Jun 13, 2019

When did it start being flaky?

@tvolkert
Copy link
Contributor

Do you have an example Cirrus task that failed, and what the output was?

@jonahwilliams
Copy link
Member Author

It was on my latest commit but someone clicked rerun, or cirrus re-ran it

@tvolkert
Copy link
Contributor

https://cirrus-ci.com/task/6705135426732032

00:19 +144: test\asset_bundle_test.dart: AssetBundle.build handle removal of wildcard directories                                                                                                      Expected: <true>
  Actual: <false>

#0      fail (package:test_api/src/frontend/expect.dart:152:30)
#1      _expect (package:test_api/src/frontend/expect.dart:146:3)
#2      expect (package:test_api/src/frontend/expect.dart:59:3)
#3      main.<anonymous closure>.<anonymous closure> (org-dartlang-app:///test/asset_bundle_test.dart:131:7)
<asynchronous suspension>
#4      testUsingContext.<anonymous closure>.<anonymous closure>.<anonymous closure>.<anonymous closure>.<anonymous closure> (org-dartlang-app:///test/src/context.dart:108:42)
<asynchronous suspension>
#5      AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:153:29)
<asynchronous suspension>
#6      _rootRun (dart:async/zone.dart:1124:13)
#7      _CustomZone.run (dart:async/zone.dart:1021:19)
#8      _runZoned (dart:async/zone.dart:1516:10)
#9      runZoned (dart:async/zone.dart:1463:12)
#10     AppContext.run (package:flutter_tools/src/base/context.dart:152:18)
<asynchronous suspension>
#11     testUsingContext.<anonymous closure>.<anonymous closure>.<anonymous closure>.<anonymous closure> (org-dartlang-app:///test/src/context.dart:96:30)
#12     _rootRun (dart:async/zone.dart:1124:13)
#13     _CustomZone.run (dart:async/zone.dart:1021:19)
#14     _runZoned (dart:async/zone.dart:1516:10)
#15     runZoned (dart:async/zone.dart:1500:12)
#16     testUsingContext.<anonymous closure>.<anonymous closure>.<anonymous closure> (org-dartlang-app:///test/src/context.dart:94:18)
#17     AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:153:29)
<asynchronous suspension>
#18     _rootRun (dart:async/zone.dart:1124:13)
#19     _CustomZone.run (dart:async/zone.dart:1021:19)
#20     _runZoned (dart:async/zone.dart:1516:10)
#21     runZoned (dart:async/zone.dart:1463:12)
#22     AppContext.run (package:flutter_tools/src/base/context.dart:152:18)
<asynchronous suspension>
#23     testUsingContext.<anonymous closure>.<anonymous closure> (org-dartlang-app:///test/src/context.dart:69:22)
#24     AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:153:29)
<asynchronous suspension>
#25     _rootRun (dart:async/zone.dart:1124:13)
#26     _CustomZone.run (dart:async/zone.dart:1021:19)
#27     _runZoned (dart:async/zone.dart:1516:10)
#28     runZoned (dart:async/zone.dart:1463:12)
#29     AppContext.run (package:flutter_tools/src/base/context.dart:152:18)
<asynchronous suspension>
#30     runInContext (package:flutter_tools/src/context_runner.dart:56:24)
<asynchronous suspension>
#31     testUsingContext.<anonymous closure> (org-dartlang-app:///test/src/context.dart:68:11)
<asynchronous suspension>
#32     Declarer.test.<anonymous closure>.<anonymous closure>.<anonymous closure> (package:test_api/src/backend/declarer.dart:168:27)
<asynchronous suspension>
#33     Invoker.waitForOutstandingCallbacks.<anonymous closure> (package:test_api/src/backend/invoker.dart:250:15)
<asynchronous suspension>
#34     _rootRun (dart:async/zone.dart:1124:13)
#35     _CustomZone.run (dart:async/zone.dart:1021:19)
#36     _runZoned (dart:async/zone.dart:1516:10)
#37     runZoned (dart:async/zone.dart:1463:12)
#38     Invoker.waitForOutstandingCallbacks (package:test_api/src/backend/invoker.dart:247:5)
#39     Declarer.test.<anonymous closure>.<anonymous closure> (package:test_api/src/backend/declarer.dart:166:33)
#40     _rootRun (dart:async/zone.dart:1124:13)
#41     _CustomZone.run (dart:async/zone.dart:1021:19)
#42     _runZoned (dart:async/zone.dart:1516:10)
#43     runZoned (dart:async/zone.dart:1463:12)
#44     Declarer.test.<anonymous closure> (package:test_api/src/backend/declarer.dart:165:13)
<asynchronous suspension>
#45     Invoker._onRun.<anonymous closure>.<anonymous closure>.<anonymous closure>.<anonymous closure> (package:test_api/src/backend/invoker.dart:400:25)
<asynchronous suspension>
#46     new Future.<anonymous closure> (dart:async/future.dart:176:37)
#47     StackZoneSpecification._run (package:stack_trace/src/stack_zone_specification.dart:209:15)
#48     StackZoneSpecification._registerCallback.<anonymous closure> (package:stack_trace/src/stack_zone_specification.dart:119:48)
#49     _rootRun (dart:async/zone.dart:1120:38)
#50     _CustomZone.run (dart:async/zone.dart:1021:19)
#51     _CustomZone.runGuarded (dart:async/zone.dart:923:7)
#52     _CustomZone.bindCallbackGuarded.<anonymous closure> (dart:async/zone.dart:963:23)
#53     StackZoneSpecification._run (package:stack_trace/src/stack_zone_specification.dart:209:15)
#54     StackZoneSpecification._registerCallback.<anonymous closure> (package:stack_trace/src/stack_zone_specification.dart:119:48)
#55     _rootRun (dart:async/zone.dart:1124:13)
#56     _CustomZone.run (dart:async/zone.dart:1021:19)
#57     _CustomZone.bindCallback.<anonymous closure> (dart:async/zone.dart:947:23)
#58     Timer._createTimer.<anonymous closure> (dart:async-patch/timer_patch.dart:21:15)
#59     _Timer._runTimers (dart:isolate-patch/timer_impl.dart:382:19)
#60     _Timer._handleMessage (dart:isolate-patch/timer_impl.dart:416:5)
#61     _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:172:12)


00:19 +144 -1: test\asset_bundle_test.dart: AssetBundle.build handle removal of wildcard directories [E]                                                                                               
  Expected: <true>
    Actual: <false>
  
  package:test_api                                       expect
  org-dartlang-app:///test/asset_bundle_test.dart 131:7  main.<fn>.<fn>
  ===== asynchronous gap ===========================
  dart:async                                             _AsyncAwaitCompleter.completeError
  org-dartlang-app:///test/asset_bundle_test.dart        main.<fn>.<fn>
  ===== asynchronous gap ===========================
  dart:async                                             _asyncThenWrapperHelper
  org-dartlang-app:///test/asset_bundle_test.dart        main.<fn>.<fn>
  org-dartlang-app:///test/src/context.dart 108:42       testUsingContext.<fn>.<fn>.<fn>.<fn>.<fn>

@tvolkert
Copy link
Contributor

I looked at the last 100 Cirrus runs, and this failed on 4 of them - all indications are that it's been flaky before that. So we're looking at ~4% flakiness potentially since the test was written.

tvolkert added a commit to tvolkert/flutter that referenced this issue Jun 14, 2019
@tvolkert
Skip flaky test on Windows
b59fc82 
flutter#34446
@tvolkert tvolkert mentioned this issue Jun 14, 2019
Merged
@tvolkert
Copy link
Contributor

Skipping the test on Windows, and removing the TODAY label.

tvolkert added a commit that referenced this issue Jun 14, 2019
@tvolkert
Skip flaky test on Windows (#34464)
530824f 
#34446
kiku-jw pushed a commit to kiku-jw/flutter that referenced this issue Jun 14, 2019
@tvolkert @kiku-jw
Skip flaky test on Windows (flutter#34464)
67b4c96 
flutter#34446
@gspencergoog
Copy link
Contributor

gspencergoog commented Oct 11, 2019 •
edited
Loading

Hmm. this started failing very consistently again today. Failed three times in a row, and reverting the first PR didn't stop the failing:

https://cirrus-ci.com/task/6014762248306688?command=test_all#L570

@Hixie Hixie added this to the Goals milestone Oct 11, 2019
@dnfield
Copy link
Contributor

dnfield commented Oct 11, 2019

I'm pretty sure this is flaky because it's trying to compare timestamps that end up not being what's expected.

https://github.com/flutter/flutter/blob/master/packages/flutter_tools/lib/src/asset.dart#L103 in particular is looking at the stat of the file in a MemoryFileSystem. It's possible that the file writes are happening quickly enough where it's resolving to the same time (instead of some time later).

@tvolkert
Copy link
Contributor

dart-archive/file.dart#129 was merged yesterday to allow this to be fixed - published in package:file version 5.1.0.

@dnfield
Copy link
Contributor

dnfield commented Oct 11, 2019

@tvolkert - that actually just makes the failure 100% instead of flakily dependent on the clock. We'd still end up with an implementation that wants to compare a timestamp of when a method got called with the stat of a file, which .. probably needs to change. I'm going to mark the test as skipped again to unblock the tree. @jonahwilliams says he's going to work on this some more on Monday.

@dnfield dnfield mentioned this issue Oct 11, 2019
Merged
@jonahwilliams jonahwilliams self-assigned this Oct 11, 2019
@dnfield dnfield removed the ⚠ TODAY label Oct 11, 2019
@jonahwilliams jonahwilliams mentioned this issue Oct 12, 2019
Merged
@github-actions
Copy link

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 13, 2021
@flutter-triage-bot flutter-triage-bot bot added the c: flake Tests that sometimes, but not always, incorrectly pass label Jul 7, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@jonahwilliams jonahwilliams

Labels
c: flake Tests that sometimes, but not always, incorrectly pass
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Deflake wildcard asset test jonahwilliams/flutter
5 participants
@Hixie @dnfield @gspencergoog @jonahwilliams @tvolkert