Don't destroy all DNS records when GKE Gateway is unhealthy because of one bad HTTPRoute

[TylerJGabb]

What would you like to be added:
I encountered #4023 while working with external-dns on GKE using the k8s Gateway API.

I noticed the following behavior:

Assume we have a gateway My-Gateway in k8s, with the ip address 1.2.3.4.

Now, lets say there are 2 HTTPRoutes attached to that gateway HTTPRoute-Foo and HTTPRoute-Bar. All routes are Healthy, and so the Gateway is healthy in GKE. external-dns adds two dns entries foo.mydomain.com and bar.mydomain.com resolving to the ip 1.2.3.4

We add a third HTTPRoute HTTPRoute-Bad, that due to some arbitrary reason, can not be resolved, and so is marked as Unhealthy or Error. As a consequence, the GKE Gateway controller marks the ENTIRE Gateway as unhealthy, and, due to this, external-dns deletes all DNS entries for the gateway, even though only one HTTPRoute is bad.

My request is to add a new flag to external dns, something like --preserve-dns-for-healthy-gateway-routes, that will make it so external-dns does not delete the dns entries of an unhealthy gateway for healthy attached routes.

Why is this needed:
In our stack, we are using a single gateway with hundreds of routes. If we add an HTTPRoute and that route is faulty for some arbitrary reason, external-dns will bring our whole stack down, because it will delete all DNS entries. We need this feature so that our DNS Records are preserved for healthy routes, even if the gateway is marked as unhealthy.

I would like to do this if possible!

[abursavich]

@TylerJGabb, please see #4023 (comment)

Getting the states of the Gateway and HTTPRoutes when this issue occurs would be very helpful.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle rotten
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

[k8s-ci-robot]

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.