-
Notifications
You must be signed in to change notification settings - Fork 374
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Increase benchmark validation timeout #4225
base: master
Are you sure you want to change the base?
Conversation
Datadog ReportBranch report: ✅ 0 Failed, 22100 Passed, 1457 Skipped, 5m 7.47s Total Time |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #4225 +/- ##
==========================================
- Coverage 97.75% 97.74% -0.01%
==========================================
Files 1355 1355
Lines 82007 82008 +1
Branches 4207 4207
==========================================
- Hits 80164 80161 -3
- Misses 1843 1847 +4 ☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
-1 flaky test! 🎉
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍 +1 Thanks for hunting down this one :)
@@ -16,7 +16,7 @@ | |||
benchmarks_to_validate.each do |benchmark| | |||
describe benchmark do | |||
it 'runs without raising errors' do | |||
expect_in_fork do | |||
expect_in_fork(timeout_seconds: 30) do |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I encountered a timeout failure in #4207 but I thought my benchmark was taking too long (which still may be the case).
My understanding is the current timeout is 10 seconds, perhaps raising it to 20 would be sufficient rather than going to 30? What do you think?
I got this flaky test recently:
It looks like this benchmark validation test has a timeout of 10 seconds that was just exceeded in the MacOS CI.
This PR increases it to 30 seconds.
Change log entry
None.