Add IPv6/dual-stack support #350

alpeb · 2024-03-11T19:47:24Z

Flags Changes

This adds the proxy-init flag --iptables-mode (with possible values legacy and nft), which supersedes --firewal-bin-path and firewall-save-bin-path (which still remain supported).
Also the --ipv6 flag has been added (default true).

After the set of rules run via iptables are processed, if --ipv6 is true (which is the default), the same set of rules will be run via ip6tables.

Analog changes were applied to linkerd-cni as well.

Backwards-Compatibility

This is backwards-compatible with older control planes and upcoming control planes.
If --ipv6 is not passed (and thus defaults to true), this doesn't impact operation even if the cluster doesn't support IPv6; the ip6tables rules are applied but they're innocuous.
OTOH if there's no kernel support for IPv6 (which is the case for github runners*) then the ip6tables command will fail but we'll just log the failure and not fail the linkerd-init container (nor the add command for linkerd-cni). This avoids having to explicitly set --ipv6=false, but it can be set if the user is aware of such limitations and wants to get rid of the errors.

Testing Improvements

The cni-plugin-integration workflow has been simplified by using a matrix strategy, and enhanced by parameterizing the iptables-mode config.

Linkerd IPv6 Support

This allows routing IPv6 traffic to the proxy, but is just the first step towards IPv6/dual-stack support. Control plane and proxy changes will come up next.

(*) Github Runners IPv6 Support

Even though modinfo signals support for IPv6, ip6tables commands throw modprobe errors. Indeed, according to actions/runner-images#668 support is not there yet.
Also, according to actions/runner#3138 there are issues with hosted runners as well, but that might not affect us if we still expose an IPv4 interface to interact with github. Something to take into account when we get to IPv6 integration testing.

Followup to linkerd/linkerd2-proxy-init#350 In support of the new proxy-init flags `--iptables-mode` and `--ipv6`: - For the linkerd-control-plane chart added the values.yaml entry `enableIPv6` (defaults to true). The `proxyInit.iptablesMode` was already there, but we interpret it now slightly differently in `_proxy-init.tpl`. - For the linkerd2-cni chart added the entries `iptablesMode` (defaults to "legacy") and `enableIPv6` (defaults to true). Note this allows routing IPv6 traffic to the proxy, but it's just the first step towards IPv6/dual-stack support. More control plane and proxy changes will come up next. *Do not merge yet*: We're pulling the images `ghcr.io/alpeb/proxy-init:ipv6` and `ghcr.io/alpeb/cni-plugin:ipv6` as temporary builds for linkerd/linkerd2-proxy-init#350, while that gets released.

mateiidavid

This looks great! Such an exciting change :D I left a few comments.

NOTE: as you're probably aware of, I have no way of testing this atm since my env's been reduced to nothingness.

cni-plugin/integration/manifests/flannel/linkerd-cni.yaml

proxy-init/cmd/root.go

cni-plugin/main.go

mateiidavid · 2024-03-13T12:59:17Z

proxy-init/cmd/root.go

 	return cmd
 }

 // BuildFirewallConfiguration returns an iptables FirewallConfiguration suitable to use to configure iptables.
-func BuildFirewallConfiguration(options *RootOptions) (*iptables.FirewallConfiguration, error) {
+func BuildFirewallConfiguration(options *RootOptions, ipv6 bool) (*iptables.FirewallConfiguration, error) {


Hm qq here: if root options already have IPv6 is it necessary to take it as a value in the function?

See my comment above.

Followup to linkerd/linkerd2-proxy-init#350 In support of the new proxy-init flags `--iptables-mode` and `--ipv6`: - For the linkerd-control-plane chart added the values.yaml entry `enableIPv6` (defaults to true). The `proxyInit.iptablesMode` was already there, but we interpret it now slightly differently in `_proxy-init.tpl`. - For the linkerd2-cni chart added the entries `iptablesMode` (defaults to "legacy") and `enableIPv6` (defaults to true). Note this allows routing IPv6 traffic to the proxy, but it's just the first step towards IPv6/dual-stack support. More control plane and proxy changes will come up next. *Do not merge yet*: We're pulling the images `ghcr.io/alpeb/proxy-init:ipv6` and `ghcr.io/alpeb/cni-plugin:ipv6` as temporary builds for linkerd/linkerd2-proxy-init#350, while that gets released.

In support of the new proxy-init flags `--iptables-mode` and `--ipv6`: - For the linkerd-control-plane chart added the values.yaml entry `enableIPv6` (defaults to true). The `proxyInit.iptablesMode` was already there, but we interpret it now slightly differently in `_proxy-init.tpl`. - For the linkerd2-cni chart added the entries `iptablesMode` (defaults to "legacy") and `enableIPv6` (defaults to true). Note this allows routing IPv6 traffic to the proxy, but it's just the first step towards IPv6/dual-stack support. More control plane and proxy changes will come up next.

mateiidavid

Left a quick comment on how I think we could refactor the options. We could copy it by dereferencing the value. I put together an example to illustrate what I mean:

https://go.dev/play/p/7GqtsFQesqV

cni-plugin/main.go

alpeb · 2024-03-14T21:08:24Z

Hey @mateiidavid thanks for the detailed review 🌮
My latest push takes into account your API suggestions, let me know if that makes sense :-)

mateiidavid

LGTM! Thanks for incorporating the feedback. I'm super happy with how this turned out and excited to get it going :D

cni-plugin/main.go

## Flags Changes This replaces the proxy-init flags `--firewall-bin-path` and `--firewall-save-bin-path` with the flag `--iptables-mode` (with possible values `legacy` and `nft`). Also the `--ipv6` flag has been added (default `true`). Proxy-init won't be relying just on the iptables commands family (iptables-legacy, iptables-legacy-save, iptables-nft, iptable-nft-save), but also on the ip6tables command family, so it's better to know the mode we're in (legacy or nft) and whether ipv6 is enabled, to determine all the commands that need to be used instead of directly passing them as arguments. After the set of rules run via iptables are processed, if `--ipv6` is true (which is the default), the same set of rules will be run via ip6tables. Analog changes were applied to linkerd-cni as well. ## Backwards-Compatibility This is backwards-compatible with older control planes as long as the older flags are not used. If those flags are used, an explanatory error is thrown (better than showing a deprecation message and failing later when legacy/nft iptables don't work). If `--ipv6` is not passed (and thus defaults to true), this doesn't impact operation even if the cluster doesn't support IPv6; the ip6tables rules are applied but they're innocuous. OTOH if there's no kernel support for IPv6 then the ip6tables command will fail but we'll just log the failure and not fail the linkerd-init container (nor the `add` command for linkerd-cni). This avoids having to explicitly set `--ipv6=false`, but it can be set if the user is aware of such limitations and wants to get rid of the errors. ## Linkerd IPv6 Support This allows routing IPv6 traffic to the proxy, but is just the first step towards IPv6/dual-stack support. Control plane and proxy changes will come up next.

alpeb · 2024-03-19T22:14:47Z

After going over this off-github we realized it'd be best to have the upcoming control plane updates remain compatible with older proxy-inits, so in d156e1a I've brought back the --firewal-bin-path and firewall-save-bin-path flags, that when set, we derive the --iptables-mode value accordingly. I've updated the description above as well to reflect this.

- For the linkerd-control-plane chart added the values.yaml entry `disableIPv6`, defaulting to false. If we wan't to explicitly set `disableIPv6=true`, the proxy-init v2.3.0 (containing linkerd/linkerd2-proxy-init#350) needs to be used, otherwise proxy-init will error out because of the unknown flag. - For the linkerd2-cni chart added the entries `iptablesMode` (defaults to "legacy") and `disableIPv6` (defaults to false). These flags can still be passed to the current proxy-init version (without linkerd/linkerd2-proxy-init#350) and will simply be ignored. Note this allows routing IPv6 traffic to the proxy by default, but it's just the first step towards IPv6/dual-stack support. More control plane and proxy changes will come next.

Followup to linkerd/linkerd2-proxy-init#350 - For the linkerd-control-plane chart added the values.yaml entry `disableIPv6`, defaulting to false. - For the linkerd2-cni chart added the entries `iptablesMode` (defaults to "legacy") and `disableIPv6` (defaults to false). Note this allows routing IPv6 traffic to the proxy by default, but it's just the first step towards IPv6/dual-stack support. More control plane and proxy changes will come next. Co-authored-by: Oliver Gould <[email protected]>

Followup to linkerd/linkerd2-proxy-init#350 - For the linkerd-control-plane chart added the values.yaml entry `disableIPv6`, defaulting to false. - For the linkerd2-cni chart added the entries `iptablesMode` (defaults to "legacy") and `disableIPv6` (defaults to false). Note this allows routing IPv6 traffic to the proxy by default, but it's just the first step towards IPv6/dual-stack support. More control plane and proxy changes will come next. Co-authored-by: Oliver Gould <[email protected]> Signed-off-by: Mark S <[email protected]>

alpeb requested a review from a team as a code owner March 11, 2024 19:47

alpeb assigned mateiidavid Mar 11, 2024

alpeb force-pushed the alpeb/ip6tables branch 2 times, most recently from 1e795ad to 7f5007b Compare March 11, 2024 22:48

alpeb changed the title ~~Add IPv6/dual-stack support~~ WIP: Add IPv6/dual-stack support Mar 11, 2024

alpeb force-pushed the alpeb/ip6tables branch from cf5f748 to 88f2a9f Compare March 12, 2024 11:51

alpeb changed the title ~~WIP: Add IPv6/dual-stack support~~ Add IPv6/dual-stack support Mar 12, 2024

alpeb mentioned this pull request Mar 12, 2024

Use IPv6-enabled proxy-init linkerd/linkerd2#12249

Merged

mateiidavid reviewed Mar 13, 2024

View reviewed changes

alpeb force-pushed the alpeb/ip6tables branch 2 times, most recently from 628b5ed to 94489ef Compare March 14, 2024 15:09

mateiidavid reviewed Mar 14, 2024

View reviewed changes

cni-plugin/main.go Outdated Show resolved Hide resolved

mateiidavid approved these changes Mar 15, 2024

View reviewed changes

cni-plugin/main.go Show resolved Hide resolved

alpeb added 4 commits March 19, 2024 15:26

matrix test for legacy/nft

6163337

Avoid passing extra ipv6 flag around

03a7172

Re-add support for firewall-bin-path, firewall-save-bin-path

d156e1a

alpeb force-pushed the alpeb/ip6tables branch from edc6086 to d156e1a Compare March 19, 2024 21:57

mateiidavid approved these changes Mar 28, 2024

View reviewed changes

alpeb merged commit 94256af into main Mar 28, 2024
21 checks passed

alpeb deleted the alpeb/ip6tables branch March 28, 2024 15:11

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add IPv6/dual-stack support #350

Add IPv6/dual-stack support #350

alpeb commented Mar 11, 2024 •

edited

Loading

mateiidavid left a comment

mateiidavid Mar 13, 2024

alpeb Mar 13, 2024

mateiidavid left a comment

alpeb commented Mar 14, 2024

mateiidavid left a comment

alpeb commented Mar 19, 2024

Add IPv6/dual-stack support #350

Add IPv6/dual-stack support #350

Conversation

alpeb commented Mar 11, 2024 • edited Loading

Flags Changes

Backwards-Compatibility

Testing Improvements

Linkerd IPv6 Support

(*) Github Runners IPv6 Support

mateiidavid left a comment

Choose a reason for hiding this comment

mateiidavid Mar 13, 2024

Choose a reason for hiding this comment

alpeb Mar 13, 2024

Choose a reason for hiding this comment

mateiidavid left a comment

Choose a reason for hiding this comment

alpeb commented Mar 14, 2024

mateiidavid left a comment

Choose a reason for hiding this comment

alpeb commented Mar 19, 2024

alpeb commented Mar 11, 2024 •

edited

Loading