-
Notifications
You must be signed in to change notification settings - Fork 36
Move to GH actions #58
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
@@ -53,6 +53,7 @@ final class GaugeTests: XCTestCase { | |||
""") | |||
} | |||
|
|||
#if os(Linux) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not super happy with this solution, but from some checking the MacOS runners make this sleep last somewhere between 0.05 and 0.1 seconds, which makes these tests incredibly flaky. Locally they run just fine.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Any input appreciated. CC @ktoso
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can introduce the concept of time dilation. We used to do this all the time in Akka where we had very slow CI machines and had to adjust timeouts on them for all tests. It's documented a bit here: https://doc.akka.io/docs/akka/current/testing.html#accounting-for-slow-test-systems
This means that we'd do: delay.dilated
where dilated would be
var dilated: Double {
if <DETECT IF ON SLOW RUNTIME>
return self * /*dilationFactor (e.g: */ 2 // easiest to have this be set in an env var
else
return self
}
So in the CI, env we'd just set the TEST_TIME_DILATION_FACTOR=10 or something that makes it happy :-)
WDYT?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If I understand correctly this would turn this test from sleeping for 0.05 seconds, to sleeping for 50. Which would than require to also update the asserts based on the time dilation, correct? In that case, would that not only make the problem worse?
In this case it's not a timeout that's messing things up, it's measuring the time a Thread.sleep
takes, and on slow CI machines it takes longer than it's supposed to, resulting in "wrong" metrics.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah I wasn't all to clear I guess. In such tests we need to adjust the "expectation" rather than the sleep -- so we'd apply the dilated to the expectation that the time was smaller than something.
All such sleepy tests are pretty meh so let's just give it a good best effort :-)
@ktoso Decided on a new strategy to do these time based tests. Thoughts? 😄 |
Move to GH Actions instead of Circle CI. Also adds Swift 5.3 and 5.4 as test targets.
Checklist