Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add thread-safe progress meter #334

Draft
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

younes-io
Copy link

No description provided.

@younes-io
Copy link
Author

@timholy could you please take a look at this?
thank you

@MarcMush
Copy link
Collaborator

it looks ok but is there a problem you came up across? Does this test something that wasn't tested before?

Copy link

codecov bot commented Nov 17, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 96.77%. Comparing base (807496a) to head (5a73a83).
Report is 22 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #334      +/-   ##
==========================================
+ Coverage   93.48%   96.77%   +3.29%     
==========================================
  Files           1        1              
  Lines         399      559     +160     
==========================================
+ Hits          373      541     +168     
+ Misses         26       18       -8     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@younes-io
Copy link
Author

it looks ok but is there a problem you came up across? Does this test something that wasn't tested before?

I added this to test how threads behave when they're all trying to update the same thing at once, but with different timing patterns. This can catch subtle issues that simpler tests might miss.

@MarcMush
Copy link
Collaborator

we've had problems with threading (#232 and linked discussions) and there can be more done to improve multi-threading (#333, #157)

I wrote this to try to catch unsafe threading

function test_thread(N)
p = Progress(N)
Threads.@threads for _ in 1:N
next!(p)
end
end
println("Brute-forcing thread safety... ($(Threads.nthreads()) threads)")
@time for i in 1:10^5
test_thread(Threads.nthreads())
end

I think no sleep is a worst-case, and sleep yields (cpu can do other stuff) which is not a realistic workload either

I would have no problem adding these tests but i don't see what they add. If you have more insight on threaded stuff please say, I'm pretty new to it

@younes-io younes-io changed the title add high contention tests feat: add thread-safe progress meter Nov 18, 2024
@younes-io
Copy link
Author

@MarcMush you're right, those tests were just simulating artificial computation.. I wasn't aware of this issue, so I pushed a commit to implement @timholy's recommendation using Threads.@spawn
Obviously, this is just a start.. Please, let me know your thoughts

@younes-io younes-io marked this pull request as draft November 20, 2024 07:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants