Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Chain Exchange protocol over pubsub #807

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

masih
Copy link
Member

@masih masih commented Dec 19, 2024

Implement chain exchange protocol over pubsub as a mechanism to propagate ECChain across the network with reasonable spam protection.

To protect against spam the mechanism employs two separate caches for chains that are generally discovered across the network and the ones explicitly looked up or broadcasted by the local node. Both caches are capped LRU, where the LRU recent-ness is used as a way to prioritise chains we cache while keeping the total memory footprint fixed. This approach is not the most memory efficient but is simpler to implement as the LRU encapsulates a lot of the complexity.

The code has a lot of TODOs as places to improve or question to the reviewer. To action most of the TODOs further refactoring across the code is needed which is intended to be actioned in separate commits.

The code path introduced here is not integrated into F3 host; future PRs will iteratively integrate the mechanism across F3 host and other places.

Part of #792

Implement chain exchange protocol over pubsub as a mechanism to
propagate `ECChain` across the network with reasonable spam protection.

To protect against spam the mechanism employs two separate caches for
chains that are generally discovered across the network and the ones
explicitly looked up or broadcasted by the local node. Both caches are
capped LRU, where the LRU recent-ness is used as a way to prioritise
chains we cache while keeping the total memory footprint fixed. This
approach is not the most memory efficient but is simpler to implement
as the LRU encapsulates a lot of the complexity.

The code has a lot of TODOs as places to improve or question to the
reviewer. To action most of the TODOs further refactoring across the
code is needed which is intended to be actioned in separate commits.

The code path introduced here is not integrated into F3 host; future PRs
will iteratively integrate the mechanism across F3 host and other
places.

Part of #792
@masih masih force-pushed the masih/pubsub-chainexchange branch from 58ce9f1 to 7f3d3ae Compare December 19, 2024 17:16
@masih masih marked this pull request as ready for review December 19, 2024 17:22
@masih masih requested a review from Kubuxu December 19, 2024 17:22
Copy link

codecov bot commented Dec 19, 2024

Codecov Report

Attention: Patch coverage is 50.68493% with 144 lines in your changes missing coverage. Please review.

Project coverage is 68.63%. Comparing base (d4f3a0c) to head (7f3d3ae).

Files with missing lines Patch % Lines
chainexchange/pubsub.go 56.93% 77 Missing and 13 partials ⚠️
chainexchange/options.go 34.14% 47 Missing and 7 partials ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #807      +/-   ##
==========================================
- Coverage   69.38%   68.63%   -0.75%     
==========================================
  Files          78       81       +3     
  Lines        7957     8249     +292     
==========================================
+ Hits         5521     5662     +141     
- Misses       1971     2101     +130     
- Partials      465      486      +21     
Files with missing lines Coverage Δ
chainexchange/chainexchange.go 100.00% <100.00%> (ø)
internal/psutil/psutil.go 32.25% <ø> (ø)
chainexchange/options.go 34.14% <34.14%> (ø)
chainexchange/pubsub.go 56.93% <56.93%> (ø)

... and 3 files with indirect coverage changes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant