Source Hydrator hydrateTo triggers hydrates every few second

[procinger]

Checklist:

• I've searched in the docs and FAQ for my answer: https://bit.ly/argocd-faq.
• I've included steps to reproduce the bug.
• I've pasted the output of argocd version.

Describe the bug
ArgoCD 2.14.8+a7178be
Helm Chart 7.8.14

I am currently testing the new feature of the source hydrator in a playground repository. So far this application manifest works without any problems

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: source-hydrator
  namespace: argo-cd
spec:
  destination:
    server: https://foo.svc
    namespace: source-hydrator
  project: default
  sourceHydrator:
    drySource:
      path: kustomize
      repoURL: [email protected]/foo
      targetRevision: HEAD
    syncSource:
      path: .
      targetBranch: environment/dev

As soon as i add the hydrateTo to the manifest, argocd tries to hydrate the branch every ~5 second and i get everytime an error message in the bottom right corner.

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: source-hydrator
  namespace: argo-cd
spec:
  destination:
    server: https://foo.svc
    namespace: source-hydrator
  project: default
  sourceHydrator:
    drySource:
      path: kustomize
      repoURL: [email protected]/foo
      targetRevision: HEAD
    syncSource:
      path: .
      targetBranch: environment/dev
    hydrateTo:
      targetBranch: environment/dev-next

I followed the steps from this guide and activated the commit server in an existing argocd installation.

Expected behavior

It should only hydrate if changes are really present and without an error message

Version

argocd-server: v2.14.8+a7178be
  BuildDate: 2025-03-24T20:54:41Z
  GitCommit: a7178be1c1ebe08a61e326037173d2ef979cb3f8
  GitTreeState: clean
  GoVersion: go1.23.3
  Compiler: gc
  Platform: linux/arm64
  Kustomize Version: v5.4.3 2024-07-19T16:40:33Z
  Helm Version: v3.16.3+gcfd0749
  Kubectl Version: v0.31.0
  Jsonnet Version: v0.20.0

[crenshaw-dev]

I think this is two bugs.

The first bug is the little error message popping up. The UI component that displays the hydrated SHA doesn't work when the input is HEAD. The UI component should handle the HEAD case correctly.

The second bug is the refreshing every 5 seconds. I know there are issues with caching in the hydrator, but nothing that should cause a hydration every 5s. Can you check the logs to see if the hydrator is logging why it's refreshing? Those logs should be in the application-controller component and should have the word "hydrator" somewhere in the log lines.

[procinger]

After a quick check of the controller I found these logs and it appears every 30 seconds. Sorry seemed faster to me when testing in the Web UI.

time="2025-03-27T15:29:03Z" level=info msg="Hydrating app" app-namespace=argo-cd app-qualified-name=argo-cd/source-hydrator application=source-hydrator project=default reason="hydration expired"

[crenshaw-dev]

Hm. That number today is based on the app refresh interval (although long-term it probably should be its own config item). Do you have that configured to 30s?

[procinger]

Yes that will be it, timeout.reconciliation has a value of 30s

[crenshaw-dev]

Gotcha. There's definitely more work avoidance that needs to be done there. I explicitly disabled some manifest-related caches (like the refs cache) just to get things running. We need to go back and see how much caching we can re-enable without hitting issues.