Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add basic sp selection score modelling #298

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

rvagg
Copy link
Member

@rvagg rvagg commented Jun 7, 2023

Getting this work up in a Draft for now so I can pause and think about something else for a little while.

This is an attempt to create a simulation of a set of retrievals in such a way that we can tweak the scoring mechanism (weights etc.) to see how/if they make a difference.

This works, (run main in pkg/session/model/cmd), but the averaging of provider behaviour across runs is causing any tweaking to get lost in the wash. I need to set up a static list of providers with fixed(ish?) characteristics up front so they can be sorted in more meaningful ways and their score comparisons have more impact.

@codecov-commenter
Copy link

Codecov Report

Merging #298 (1bcd218) into main (fe9e760) will decrease coverage by 3.29%.
The diff coverage is 3.28%.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #298      +/-   ##
==========================================
- Coverage   72.55%   69.27%   -3.29%     
==========================================
  Files          68       72       +4     
  Lines        6217     6519     +302     
==========================================
+ Hits         4511     4516       +5     
- Misses       1458     1753     +295     
- Partials      248      250       +2     
Impacted Files Coverage Δ
pkg/session/model/cmd/main.go 0.00% <0.00%> (ø)
pkg/session/model/pop.go 0.00% <0.00%> (ø)
pkg/session/model/prob.go 0.00% <0.00%> (ø)
pkg/session/model/sim.go 0.00% <0.00%> (ø)
pkg/session/state.go 92.11% <90.90%> (+0.30%) ⬆️

... and 2 files with indirect coverage changes

@rvagg rvagg force-pushed the rvagg/score-modelling branch from 1bcd218 to f7f497d Compare June 7, 2023 09:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants