ArgoCD Troubleshooting Guide

by Bala

Comprehensive troubleshooting guide for common ArgoCD issues across different cloud providers

ArgoCD Troubleshooting Guide

This guide covers common issues and their solutions when running ArgoCD across different cloud providers.

Video Tutorial

Learn more about troubleshooting ArgoCD in this comprehensive video tutorial:

View Source Code

Common Issues

1. Application Sync Issues

Symptom: Application Stuck in “OutOfSync” State

# Check application status
kubectl get application -n argocd

# Check detailed sync status
argocd app get my-app

Solutions:

  1. Check Git Repository Access
# Test repository access
argocd repo list
argocd repo test https://github.com/org/repo.git
  1. Verify RBAC Permissions
# Check RBAC configuration
apiVersion: v1
kind: ConfigMap
metadata:
  name: argocd-rbac-cm
  namespace: argocd
data:
  policy.csv: |
    p, role:org-admin, applications, *, */*, allow
  1. Check Resource Health
# Get resource health status
argocd app resources my-app
kubectl describe application my-app -n argocd

2. Authentication Issues

Symptom: Unable to Login to ArgoCD UI

Solutions:

  1. Reset Admin Password
# Get initial admin password
kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d

# Update password
argocd account update-password
  1. Check SSO Configuration
# Verify SSO settings
apiVersion: v1
kind: ConfigMap
metadata:
  name: argocd-cm
data:
  url: https://argocd.example.com
  dex.config: |
    connectors:
      - type: github
        id: github
        name: GitHub
        config:
          clientID: aabbccddeeff00112233
          clientSecret: $dex.github.clientSecret

3. Network Connectivity Issues

Symptom: Unable to Access ArgoCD UI or API

Solutions:

  1. Check Ingress Configuration
# Verify ingress status
kubectl get ingress -n argocd
kubectl describe ingress argocd-server-ingress -n argocd
  1. Test Network Policies
# Verify network policies
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: argocd-server-network-policy
spec:
  podSelector:
    matchLabels:
      app.kubernetes.io/name: argocd-server
  ingress:
  - {}
  1. Check Service Status
# Verify service status
kubectl get svc -n argocd
kubectl describe svc argocd-server -n argocd

4. Resource Management Issues

Symptom: High Memory/CPU Usage

Solutions:

  1. Check Resource Limits
# Adjust resource limits
apiVersion: apps/v1
kind: Deployment
metadata:
  name: argocd-server
spec:
  template:
    spec:
      containers:
      - name: argocd-server
        resources:
          limits:
            cpu: 1000m
            memory: 1Gi
          requests:
            cpu: 500m
            memory: 512Mi
  1. Enable Application Resource Tracking
# Configure resource tracking
apiVersion: v1
kind: ConfigMap
metadata:
  name: argocd-cm
data:
  resource.customizations: |
    argoproj.io/Application:
      health.lua: |
        hs = {}
        hs.status = "Healthy"
        return hs

5. Cloud Provider Specific Issues

AWS EKS Issues

  1. IRSA Configuration
# Verify IAM role
aws iam get-role --role-name argocd-role

# Check pod identity
kubectl exec -it deploy/argocd-server -n argocd -- aws sts get-caller-identity
  1. ALB Issues
# Check ALB controller logs
kubectl logs -n kube-system deployment.apps/aws-load-balancer-controller

# Verify target group binding
kubectl get targetgroupbinding -n argocd

GCP GKE Issues

  1. Workload Identity
# Verify workload identity
kubectl run -it --rm debug --image=google/cloud-sdk:slim --serviceaccount=argocd-server -n argocd -- gcloud auth list

# Check IAM binding
gcloud iam service-accounts get-iam-policy argocd@PROJECT_ID.iam.gserviceaccount.com
  1. GCP Load Balancer
# Check ingress status
kubectl describe ingress argocd-server-ingress -n argocd

# Verify backend service
gcloud compute backend-services list

Azure AKS Issues

  1. Azure AD Integration
# Check pod identity status
kubectl get azureidentity -n argocd
kubectl get azureidentitybinding -n argocd

# Verify managed identity
az identity show --name argocd-identity --resource-group argocd-rg
  1. Application Gateway
# Check application gateway health
az network application-gateway show-backend-health -g argocd-rg -n argocd-appgw

# Verify routing rules
az network application-gateway rule list -g argocd-rg --gateway-name argocd-appgw

6. Performance Optimization

Symptom: Slow Application Sync

Solutions:

  1. Enable Parallel Processing
# Configure parallel processing
apiVersion: v1
kind: ConfigMap
metadata:
  name: argocd-cm
data:
  controller.parallel.limit: "10"
  1. Optimize Resource Hooks
# Configure resource hooks
apiVersion: batch/v1
kind: Job
metadata:
  name: backup-job
  annotations:
    argocd.argoproj.io/hook: PreSync
    argocd.argoproj.io/hook-delete-policy: HookSucceeded

7. Backup and Recovery

Symptom: Data Loss or Corruption

Solutions:

  1. Configure Backup
# Setup backup
apiVersion: v1
kind: ConfigMap
metadata:
  name: argocd-cm
data:
  backup.schedule: "0 * * * *"
  backup.retention: "720h"
  1. Restore from Backup
# Restore procedure
argocd admin backup restore /path/to/backup.db

# Verify restoration
argocd app list

Diagnostic Commands

1. Application Diagnostics

# Get application status
argocd app get my-app

# Check sync history
argocd app history my-app

# View logs
argocd app logs my-app

2. Cluster Health

# Check cluster status
argocd cluster list

# Verify cluster resources
kubectl get pods -n argocd
kubectl top pods -n argocd

3. Repository Management

# List repositories
argocd repo list

# Test repository connection
argocd repo test https://github.com/org/repo.git

Best Practices for Prevention

  1. Regular Monitoring

    • Set up alerts for sync failures
    • Monitor resource usage
    • Track application health

  2. Backup Strategy

    • Regular backups
    • Test restore procedures
    • Document recovery process

  3. Security Measures

    • Regular secret rotation
    • RBAC review
    • Network policy audit

  4. Resource Management

    • Set appropriate limits
    • Configure auto-scaling
    • Monitor resource usage

Common Error Messages and Solutions

Error MessagePossible CauseSolution
”Unable to sync”Git repo issuesCheck credentials and connectivity
”Connection refused”Network policyReview network policies
”Permission denied”RBAC issuesCheck RBAC configuration
”Out of memory”Resource limitsAdjust resource limits

Logging and Debugging

1. Enable Debug Logging

# Configure logging
apiVersion: v1
kind: ConfigMap
metadata:
  name: argocd-cm
data:
  logging.level: debug
  logging.format: json

2. Log Collection

# Collect logs
kubectl logs -n argocd -l app.kubernetes.io/name=argocd-server
kubectl logs -n argocd -l app.kubernetes.io/name=argocd-repo-server
kubectl logs -n argocd -l app.kubernetes.io/name=argocd-application-controller

Conclusion

When troubleshooting ArgoCD:

  1. Identify the specific component causing issues
  2. Check logs and error messages
  3. Use appropriate diagnostic commands
  4. Follow cloud provider specific guidelines
  5. Document solutions for future reference

Additional Resources