Disable flaky iOS Scenario app tests #21080
Conversation
cbracken
force-pushed
the
disable-scenario-app-tests
branch
3 times, most recently
from
36a8d58 to
b37d328
Compare
|
Done. PTAL. |
|
Thanks. Sorry for the trouble |
This disables the macOS Scenarios app tests until a fix for the current flakiness is found. This also reverts commit 55d447a where one test was previously disabled. However, another started failing soon after, so instead we disable the whole suite here.
cbracken
force-pushed
the
disable-scenario-app-tests
branch
from
b37d328 to
8048741
Compare
|
When I just disable the tests in the project, the presubmit seems to hang indefinitely :( I've reverted back to the original hacky approach of just commenting the whole thing out until we've got a fix so that we can get the upstream rollers unblocked. |
|
Sorry for the trouble, but could we just disable the individual tests that are flaky? e.g. |
|
I'd really not rather play whack-a-mole. The last time we disabled one, we hit another one instead. The engine tree has been closed for over 5 hours at this point, which is blocking Skia, Dart, and Fuchsia rollers, and causing commits to pile up. Each time I redo/re-push this patch we restart the ~40 minute presubmit, delaying things further. |
|
that's unfortunate - sounds like the rolls are more critical at this point |
Getting the tree green is almost always more critical if we consider the transitive closure of folks impacted by a tree closure and the risk of huge Skia/Dart/Fuchsia rolls containing hundreds of commits that are difficult to bisect + the impact on the team when do do need to bisect them. Once I've got the tree green and the upstream rolls in, I'll try re-enabling these with that test disabled and work from there until we've got the most running we can without the flakes. |
|
FWIW I'm suspecting that maybe the pool of new mac builder hardware might be related given the timing. Not sure if these tests are timing-sensitive and if the new machines have different performance characteristics than the old ones? |
This disables the macOS Scenarios app tests until a fix for the flakiness is found.
This reverts commit 55d447a (#21075) which disabled one of the tests. After landing, the Scenarios app tests started failing on a different test.
Related issue: flutter/flutter#61620