Using address pool to expose services

[chunter0]

Hi all,

I'm wanting to expose multiple services on a single node. I'm running into issues here and it looks like the problem is the LoadBalancer k3d deploys uses a single static ip address to expose any LoadBalancer service type. Looks like it has no way to handle collisions, you can see this using the following setup.

Cluster:

cat <<- EOF | k3d cluster create -c -
  apiVersion: k3d.io/v1alpha5
  kind: Simple
  metadata:
    name: mgmt-cluster
  image: rancher/k3s:v1.26.14-k3s1
  options:    
    kubeconfig:
      updateDefaultKubeconfig: true
      switchCurrentContext: true
EOF

App: helloworld-v1

cat <<- EOF | kc apply -f -
apiVersion: v1
kind: Namespace
metadata:
  name: helloworld
---
apiVersion: v1
kind: Service
metadata:
  name: helloworld-v1
  namespace: helloworld
  labels:
    app: helloworld
    service: helloworld
spec:
  type: LoadBalancer
  ports:
  - port: 5000
    name: http
  selector:
    app: helloworld
    version: v1
---
  apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: helloworld-v1
    namespace: helloworld
    labels:
      app: helloworld
      version: v1
  spec:
    replicas: 1
    selector:
      matchLabels:
        app: helloworld
        version: v1
    template:
      metadata:
        labels:
          app: helloworld
          version: v1
      spec:
        containers:
        - name: helloworld
          image: docker.io/istio/examples-helloworld-v1:1.0
          resources:
            requests:
              cpu: "100m"
          imagePullPolicy: IfNotPresent
          ports:
          - containerPort: 5000
EOF

App: helloworld-v2

cat <<- EOF | kc apply -f -
apiVersion: v1
kind: Namespace
metadata:
  name: helloworld
---
apiVersion: v1
kind: Service
metadata:
  name: helloworld-v2
  namespace: helloworld
  labels:
    app: helloworld
    service: helloworld
spec:
  type: LoadBalancer
  ports:
  - port: 5000
    name: http
  selector:
    app: helloworld
    version: v2
---
  apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: helloworld-v2
    namespace: helloworld
    labels:
      app: helloworld
      version: v2
  spec:
    replicas: 1
    selector:
      matchLabels:
        app: helloworld
        version: v2
    template:
      metadata:
        labels:
          app: helloworld
          version: v2
      spec:
        containers:
        - name: helloworld
          image: docker.io/istio/examples-helloworld-v2:1.0
          resources:
            requests:
              cpu: "100m"
          imagePullPolicy: IfNotPresent
          ports:
          - containerPort: 5000
EOF

The following deployment results in only one service receiving an allocated IP Address. While the other is left in pending state with no information about why.

➜ kubectl get pods -n kube-system -l 'svccontroller.k3s.cattle.io/svcnamespace=helloworld'
NAME                                 READY   STATUS    RESTARTS   AGE
svclb-helloworld-v1-4c1e8375-j2zdw   1/1     Running   0          12m
svclb-helloworld-v2-4c555cef-tndn2   0/1     Pending   0          10m

➜ kubectl get pods -n helloworld -o wide
NAME                            READY   STATUS    RESTARTS   AGE   IP           NODE                        NOMINATED NODE   READINESS GATES
helloworld-v1-798c9bf66-cqwbm   1/1     Running   0          14m   10.42.0.18   k3d-mgmt-cluster-server-0   <none>           <none>
helloworld-v2-77f98b76b-cstjm   1/1     Running   0          12m   10.42.0.19   k3d-mgmt-cluster-server-0   <none>           <none>

Wondering if one solution would be to give the LoadBalancer access to assign addresses from a pool and logic to check for collisions?