FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

[WIP] hack: add opt-in NAT64 support to IPv6-only kind clusters by ygao-g · Pull Request #1275 · agent-substrate/substrate · GitHub

[WIP] hack: add opt-in NAT64 support to IPv6-only kind clusters - #1275

Draft
Yuan Gao (ygao-g) wants to merge 3 commits into
agent-substrate:mainfrom
ygao-g:kind-ipv6-nat64
Draft

[WIP] hack: add opt-in NAT64 support to IPv6-only kind clusters#1275
Yuan Gao (ygao-g) wants to merge 3 commits into
agent-substrate:mainfrom
ygao-g:kind-ipv6-nat64

Conversation

Yuan Gao (ygao-g) commented Aug 27, 2026
edited
Loading

Copy link
Copy Markdown
Collaborator

⚠️ Depends on #958: This PR is built on top of #958. Please review and merge that PR first.

Description

GitHub-hosted runners only support IPv4. When running an IPv6-only kind cluster on them, external endpoints are unreachable. This PR adds opt-in NAT64/DNS64 support to translate traffic and allow these clusters to reach the outside world.

Key Changes

  • Opt-in NAT64: Added IPV6_DNS64_PREFIX to create-kind-cluster.sh.
  • Agent Deployment: Vendored and deploys the kubernetes-sigs/nat64 agent, and configures CoreDNS for DNS64 synthesis.
  • CIDR Adjustments: Narrows the node podSubnet to /120 when enabled, due to the NAT64 agent's IPv4 pool limits.
  • Automated Probing: Runs a startup probe to fetch an external endpoint, ensuring translation works before proceeding.

Testing Performed

  • Ran the IPv6-only kind e2e workflow on amd64 GitHub runners (no IPv6 egress).
  • Minimal stack (prs: 958 1275, i.e. just this PR on its base):
    https://github.com/ygao-g/substrate/actions/runs/33929661265
    Cluster bring-up with the connectivity probe, DNS64 synthesis of
    storage.googleapis.com into 64:ff9b::/96, the full Agent Substrate
    install, and the gVisor runtime download over the IPv6-only pod network
    are all green. The run ends red at the micro-VM demo on a pre-existing
    IPv6 parity gap (unbracketed ClusterIP in the kata-asset endpoint),
    unrelated to this change.
  • Full IPv6 stack (prs: 958 753 979 1275 1122):
    https://github.com/ygao-g/substrate/actions/runs/33892185054
    41 passed / 0 failed / 0 skipped on both legs, including actor egress
    through 64:ff9b::.

Both runs used commit 872a8fb; the current head differs only by
commit-message amends (same tree).

Copy link
Copy Markdown
Collaborator

why do we need egress off of the host machine from the test cluster at all?

Copy link
Copy Markdown
Collaborator

we push images into the registry / cluster

Yuan Gao (ygao-g) commented Aug 27, 2026
edited
Loading

Copy link
Copy Markdown
Collaborator Author

why do we need egress off of the host machine from the test cluster at all?

Benjamin Elder (@BenTheElder) It's the actor-egress e2e tests, which fetch real external origins:

Before, on an IPv6-only cluster the script rewrote kind's
`https://[::1]:PORT` kubeconfig entry to `https://localhost:PORT`
unconditionally. That breaks any host whose `/etc/hosts` leaves
`localhost` off the `::1` line, including the Ubuntu cloud image Lima
runs. Remove the repoint logic, but note that limactl re-forwards the
published port to the host's v4 loopback only, so a macOS client cannot
directly access kind clusters run inside a Lima VM via `[::1]`. Instead,
operations should run inside the guest.

Tested: manual tests creating IPv6-only kind clusters on a local env
with IPv6 egress, namely macOS + Lima VM.
CoreDNS inherits the node's IPv4 resolver, which a v6-only pod cannot
reach. Change CoreDNS's Corefile to forward to an overridable IPv6
upstream. Also add a kind-registry:53 server block to CoreDNS. In
addition, `kind create` returns before the apiserver answers, so add
logic to wait for the control plane from inside the node first.

Tested: manual tests of creation and installation on a local env with
IPv6 egress, namely macOS + Lima VM. CI cannot verify this — the GitHub
runner is IPv4-only, so an IPv6-only cluster there additionally needs
DNS64 and NAT64.

Copy link
Copy Markdown
Collaborator Author

why do we need egress off of the host machine from the test cluster at all?

Benjamin Elder (Benjamin Elder (@BenTheElder)) It's the actor-egress e2e tests, which fetch real external origins:

Updates on this:

  • Discuss with others and have a pending item to use an in-cluster origin instead of example.com in the egress tests;
  • However, there is still traffic to external domain, as atelet that will fetch the runtime from gs://gviros/...

Benjamin Elder (@BenTheElder) Does this answer your questions?

Yuan Gao (ygao-g) changed the title hack: add opt-in NAT64 support to IPv6-only kind clusters [WIP] hack: add opt-in NAT64 support to IPv6-only kind clusters Sep 4, 2026
Yuan Gao (ygao-g) force-pushed the kind-ipv6-nat64 branch 2 times, most recently from 872a8fb to 4238d84 Compare September 8, 2026 17:22
Yuan Gao (ygao-g) added the DO NOT MERGE This PR must not be merged yet. label Sep 8, 2026
Yuan Gao (ygao-g) force-pushed the kind-ipv6-nat64 branch 2 times, most recently from feca14c to 90bf396 Compare September 8, 2026 18:04
GitHub-hosted runners only support IPv4. To allow IPv6-only kind
clusters to reach external services in this environment, we need
NAT64 translation.

This change introduces an `IPV6_DNS64_PREFIX` flag to opt into NAT64.
When set, it deploys the `kubernetes-sigs/nat64` agent and configures
CoreDNS and the node `podSubnet` accordingly.

Tested: via the IPv6-only kind e2e workflow on GitHub runners (which
have no IPv6 egress). Cluster creation and the full Agent Substrate
install succeeded, which includes downloading the gVisor runtime from
Google Cloud Storage.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/network DO NOT MERGE This PR must not be merged yet.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants


Back | FazBrowse Home | New Git URL