Skip to content
Mulgamulga

GitOps on EKS (Argo CD + EBS-CSI)

Deliver a Spinifex-themed app to EKS with GitOps — the Argo CD addon syncs it from a git repo, an EBS-CSI (Viperblock) PersistentVolume holds its state, and it is served over HTTPS through the AWS Load Balancer Controller + ACM, using Terraform on Spinifex.

terraformekskubernetesargocdgitopsebs-csistorageworkbook

Overview

This workbook keeps everything from EKS HTTPS Ingress — the public/private VPC, the NAT gateway, the LBC addon, the ACM cert, the HTTPS Ingress — and adds the parts you reach for once an app is real: GitOps delivery and persistent storage.

Three changes over the previous rung:

  1. Argo CD delivers the app. Instead of Terraform applying the Deployment, the argocd addon syncs the app from a git repository (mulgadc/eks-demo-app). Terraform only registers the repo credential and creates the Argo CD Application; Argo CD reconciles the manifests and self-heals drift.
  2. State lives on an EBS volume. The aws-ebs-csi-driver addon ships a default gp3 StorageClass. The app's PersistentVolumeClaim (in the git repo) dynamically provisions a Viperblock-backed EBS volume; the demo's hit counter persists to it and survives pod restarts.
  3. Access-entry RBAC. A second IAM principal is granted read-only cluster access via an access entry bound to AmazonEKSViewPolicy.

Terraform manages the cluster, the addons, the ACM cert, and the HTTPS Ingress; Argo CD manages the app's Deployment, Service, and PVC from git. The Ingress points at the Service Argo CD creates, so the app is reachable over HTTPS the moment Argo CD finishes its first sync.

What you'll learn:

  • Installing the argocd and aws-ebs-csi-driver addons through the EKS API
  • Registering a private git repo with Argo CD and driving an Application from Terraform
  • Dynamically provisioning a Viperblock-backed EBS volume with a PersistentVolumeClaim
  • Splitting ownership: Terraform owns infra + Ingress, Argo CD owns the workload
  • Granting scoped read-only access with an EKS access entry

What gets created

ResourceNamePurpose
VPC + subnetseks-gitops-*Public/private network (10.32.0.0/16) with a NAT gateway
IAM Roleseks-gitops-cluster-role, -node-role, -viewerControl-plane, worker, and read-only viewer roles
ECR Repositoryspinifex-demoHolds the demo image the workers pull
EKS Clustereks-gitopsPublic + private endpoints; managed-ingress=false
Node Groupworkersnode_desired_size t3.large worker(s) — 1 or 3
Addonsaws-load-balancer-controller, argocd, aws-ebs-csi-driverIngress, GitOps delivery, persistent storage
ACM Certificateeks-gitops-ingressSelf-signed, imported; on the HTTPS listener
Access Entry + Assoc.eks-gitops-viewerAmazonEKSViewPolicyRead-only, cluster scope
Argo CD repo Secreteks-demo-app-repoCredential for the private git repo
Argo CD Applicationspinifex-demoSyncs the app from git
K8s Ingressspinifex-demoingressClassName: alb, HTTPS via the ACM cert
Git-managed (by Argo CD)Deployment, Service, PVCThe app + its Viperblock-backed volume

Spinifex specifics

  • The argocd and aws-ebs-csi-driver bundles must be baked into the eks-node AMI — describe-addon returns no baked bundle otherwise.
  • The EBS-CSI default StorageClass uses provisioner: ebs.csi.aws.com; a PVC against it provisions a Viperblock-backed EBS volume.
  • If the PVC stays Pending, the app still serves (the counter falls back to in-memory) — see Troubleshooting.
  • Leave addon versions unset (the AWS provider and the catalog disagree on the version-string format).
  • The Argo CD Application is a kubernetes_manifest, so the argoproj.io CRDs must exist before apply — apply the parent and let the addon reach ACTIVE first.

Prerequisites:

  • Spinifex running with an eks-node image carrying the LBC, Argo CD, and EBS-CSI bundles
  • OpenTofu or Terraform, plus kubectl, the AWS CLI, and Docker
  • The demo image built and pushed to ECR (see ../demo-app/README.md)
  • A git repo with the app manifests (mulgadc/eks-demo-app); a read-only PAT if it is private

Instructions

Step 1. Get the Template

bash
git clone --depth 1 --filter=blob:none --sparse https://github.com/mulgadc/spinifex.git spinifex-tf
cd spinifex-tf
git sparse-checkout set docs/terraform-workbooks
cd docs/terraform-workbooks/eks-gitops-argocd
hcl
# Example: GitOps on EKS with Argo CD + a persistent EBS-CSI volume
#
# The top rung of the EKS ladder. It extends eks-https-ingress (LBC + ACM HTTPS)
# and changes how the app is delivered: instead of Terraform applying the
# workload, the Argo CD addon syncs a more elaborate Spinifex-themed app from a
# git repository, and the app stores state on a Viperblock-backed EBS volume
# provisioned dynamically through the EBS-CSI driver.
#
# What it adds over eks-https-ingress:
#   * The argocd addon — GitOps continuous delivery, installed via the EKS API.
#   * The aws-ebs-csi-driver addon — dynamic EBS (Viperblock) PersistentVolumes.
#   * An access entry granting a second IAM principal read-only cluster access.
#
# The workloads/ module registers the (private) git repo with Argo CD, creates an
# Argo CD Application that syncs the app from it, and keeps the HTTPS Ingress
# (LBC + ACM) pointing at the git-managed Service. The app's manifests — including
# the PersistentVolumeClaim — live in the git repo (see ../../../../eks-demo-app).
#
# Usage:
#   cd spinifex/docs/terraform-workbooks/eks-gitops-argocd
#   export AWS_PROFILE=spinifex
#   tofu init && tofu apply
#   # build + push the demo image to the ECR repo this creates (see README),
#   # then: cd workloads && tofu init && tofu apply -var git_repo_url=... -var git_token=...
#   # finally: kubectl get ingress spinifex-demo -o wide  → open https://<address>

terraform {
  required_version = ">= 1.6.0"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 5.40, < 6.0"
    }
    tls = {
      source  = "hashicorp/tls"
      version = ">= 4.0"
    }
  }
}

# ---------------------------------------------------------------------------
# Variables
# ---------------------------------------------------------------------------

variable "region" {
  type    = string
  default = "ap-southeast-2"
}

variable "cluster_name" {
  type    = string
  default = "eks-gitops"
}

variable "k8s_version" {
  type    = string
  default = "1.32"
}

variable "node_instance_type" {
  type    = string
  default = "t3.medium"
}

variable "node_desired_size" {
  type        = number
  default     = 1
  description = "Worker count. Use 1 for a single-node demo, or 3 for an HA-shaped cluster."

  validation {
    condition     = var.node_desired_size == 1 || var.node_desired_size == 3
    error_message = "node_desired_size must be 1 or 3."
  }
}

variable "node_port" {
  type        = number
  default     = 30080
  description = "NodePort the demo Service is published on and the ALB forwards to"
}

variable "argocd_node_port" {
  type        = number
  default     = 30081
  description = "NodePort the Argo CD UI Service is published on and its ALB forwards to"
}

variable "cert_common_name" {
  type    = string
  default = "eks-gitops.spinifex.local"
}

variable "api_public_access_cidr" {
  type        = string
  default     = "0.0.0.0/0"
  description = "CIDR allowed to reach the public Kubernetes API endpoint; tighten in production"
}

variable "spinifex_endpoint" {
  type    = string
  default = "https://127.0.0.1:9999"
}

# ---------------------------------------------------------------------------
# Providers
# ---------------------------------------------------------------------------

provider "aws" {
  region = var.region

  endpoints {
    ec2 = var.spinifex_endpoint
    iam = var.spinifex_endpoint
    sts = var.spinifex_endpoint
    eks = var.spinifex_endpoint
    ecr = var.spinifex_endpoint
    acm = var.spinifex_endpoint
  }

  skip_credentials_validation = true
  skip_metadata_api_check     = true
  skip_requesting_account_id  = true
  skip_region_validation      = true
}

data "aws_availability_zones" "available" {
  state = "available"
}

# ---------------------------------------------------------------------------
# VPC — public subnets for the ALB + NAT, private subnets for the workers
# ---------------------------------------------------------------------------

resource "aws_vpc" "main" {
  cidr_block           = "10.32.0.0/16"
  enable_dns_hostnames = true
  enable_dns_support   = true

  tags = {
    Name = "${var.cluster_name}-vpc"
  }
}

resource "aws_internet_gateway" "igw" {
  vpc_id = aws_vpc.main.id

  tags = {
    Name = "${var.cluster_name}-igw"
  }
}

resource "aws_subnet" "public_a" {
  vpc_id                  = aws_vpc.main.id
  cidr_block              = "10.32.1.0/24"
  availability_zone       = data.aws_availability_zones.available.names[0]
  map_public_ip_on_launch = true

  tags = {
    Name = "${var.cluster_name}-public-a"
  }
}

resource "aws_subnet" "public_b" {
  vpc_id                  = aws_vpc.main.id
  cidr_block              = "10.32.2.0/24"
  availability_zone       = data.aws_availability_zones.available.names[0]
  map_public_ip_on_launch = true

  tags = {
    Name = "${var.cluster_name}-public-b"
  }
}

resource "aws_subnet" "private_a" {
  vpc_id            = aws_vpc.main.id
  cidr_block        = "10.32.11.0/24"
  availability_zone = data.aws_availability_zones.available.names[0]

  tags = {
    Name = "${var.cluster_name}-private-a"
  }
}

resource "aws_subnet" "private_b" {
  vpc_id            = aws_vpc.main.id
  cidr_block        = "10.32.12.0/24"
  availability_zone = data.aws_availability_zones.available.names[0]

  tags = {
    Name = "${var.cluster_name}-private-b"
  }
}

resource "aws_route_table" "public" {
  vpc_id = aws_vpc.main.id

  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = aws_internet_gateway.igw.id
  }

  tags = {
    Name = "${var.cluster_name}-public-rt"
  }
}

resource "aws_route_table_association" "public_a" {
  subnet_id      = aws_subnet.public_a.id
  route_table_id = aws_route_table.public.id
}

resource "aws_route_table_association" "public_b" {
  subnet_id      = aws_subnet.public_b.id
  route_table_id = aws_route_table.public.id
}

# NAT gateway gives the private workers outbound internet to pull the demo image
# from ECR. The workers still join the cluster over the in-VPC private endpoint.
resource "aws_eip" "nat" {
  domain = "vpc"

  tags = {
    Name = "${var.cluster_name}-nat-eip"
  }
}

resource "aws_nat_gateway" "nat" {
  allocation_id = aws_eip.nat.id
  subnet_id     = aws_subnet.public_a.id

  depends_on = [aws_internet_gateway.igw]

  tags = {
    Name = "${var.cluster_name}-nat"
  }
}

resource "aws_route_table" "private" {
  vpc_id = aws_vpc.main.id

  route {
    cidr_block     = "0.0.0.0/0"
    nat_gateway_id = aws_nat_gateway.nat.id
  }

  tags = {
    Name = "${var.cluster_name}-private-rt"
  }
}

resource "aws_route_table_association" "private_a" {
  subnet_id      = aws_subnet.private_a.id
  route_table_id = aws_route_table.private.id
}

resource "aws_route_table_association" "private_b" {
  subnet_id      = aws_subnet.private_b.id
  route_table_id = aws_route_table.private.id
}

# ---------------------------------------------------------------------------
# IAM — cluster role
# ---------------------------------------------------------------------------

resource "aws_iam_role" "cluster" {
  name = "${var.cluster_name}-cluster-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect    = "Allow"
      Action    = "sts:AssumeRole"
      Principal = { Service = "eks.amazonaws.com" }
    }]
  })
}

resource "aws_iam_role_policy_attachment" "cluster" {
  role       = aws_iam_role.cluster.name
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSClusterPolicy"
}

# ---------------------------------------------------------------------------
# IAM — node role
# ---------------------------------------------------------------------------

resource "aws_iam_role" "node" {
  name = "${var.cluster_name}-node-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect    = "Allow"
      Action    = "sts:AssumeRole"
      Principal = { Service = "ec2.amazonaws.com" }
    }]
  })
}

resource "aws_iam_role_policy_attachment" "node_worker" {
  role       = aws_iam_role.node.name
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy"
}

resource "aws_iam_role_policy_attachment" "node_cni" {
  role       = aws_iam_role.node.name
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy"
}

resource "aws_iam_role_policy_attachment" "node_ecr" {
  role       = aws_iam_role.node.name
  policy_arn = "arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly"
}

# The AWS Load Balancer Controller runs with the node's instance-profile
# credentials (Spinifex wires creds at the node level, not IRSA — see the addon
# block below), so the permissions it needs to manage ALBs must live on the node
# role. This mirrors the upstream AWSLoadBalancerControllerIAMPolicy, trimmed to
# the actions an ALB Ingress exercises (Shield/WAF/Cognito are disabled on the
# controller). Resources are "*" since Spinifex evaluates grants by action.
# A customer-managed policy + attachment is used (Spinifex implements
# CreatePolicy/AttachRolePolicy, not inline PutRolePolicy).
resource "aws_iam_policy" "node_lbc" {
  name = "${var.cluster_name}-node-lbc"

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Action = [
          "ec2:DescribeAccountAttributes",
          "ec2:DescribeAddresses",
          "ec2:DescribeAvailabilityZones",
          "ec2:DescribeInternetGateways",
          "ec2:DescribeVpcs",
          "ec2:DescribeSubnets",
          "ec2:DescribeSecurityGroups",
          "ec2:DescribeInstances",
          "ec2:DescribeNetworkInterfaces",
          "ec2:DescribeTags",
          "ec2:GetCoipPoolUsage",
          "ec2:DescribeCoipPools",
          "ec2:CreateSecurityGroup",
          "ec2:CreateTags",
          "ec2:DeleteTags",
          "ec2:AuthorizeSecurityGroupIngress",
          "ec2:RevokeSecurityGroupIngress",
          "ec2:DeleteSecurityGroup",
          "elasticloadbalancing:*",
          "acm:ListCertificates",
          "acm:DescribeCertificate",
          "acm:GetCertificate",
          "iam:CreateServiceLinkedRole",
          "iam:ListServerCertificates",
          "iam:GetServerCertificate",
          "tag:GetResources",
          "tag:TagResources",
          "wafv2:GetWebACLForResource",
          "shield:GetSubscriptionState"
        ]
        Resource = "*"
      }
    ]
  })
}

resource "aws_iam_role_policy_attachment" "node_lbc" {
  role       = aws_iam_role.node.name
  policy_arn = aws_iam_policy.node_lbc.arn
}

# The EBS CSI driver also runs with the node's instance-profile credentials, so
# the volume-lifecycle permissions it needs must live on the node role too. This
# mirrors the upstream AmazonEBSCSIDriverPolicy. Resources are "*" since Spinifex
# evaluates grants by action; the CreateVolume/CreateSnapshot/CreateTags grants
# upstream scope with request/resource tag conditions, omitted here because
# Spinifex does not yet evaluate IAM tag conditions.
resource "aws_iam_policy" "node_ebs_csi" {
  name = "${var.cluster_name}-node-ebs-csi"

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Action = [
          "ec2:CreateSnapshot",
          "ec2:AttachVolume",
          "ec2:DetachVolume",
          "ec2:ModifyVolume",
          "ec2:DescribeAvailabilityZones",
          "ec2:DescribeInstances",
          "ec2:DescribeSnapshots",
          "ec2:DescribeTags",
          "ec2:DescribeVolumes",
          "ec2:DescribeVolumesModifications",
          "ec2:CreateTags",
          "ec2:DeleteTags",
          "ec2:CreateVolume",
          "ec2:DeleteVolume",
          "ec2:DeleteSnapshot"
        ]
        Resource = "*"
      }
    ]
  })
}

resource "aws_iam_role_policy_attachment" "node_ebs_csi" {
  role       = aws_iam_role.node.name
  policy_arn = aws_iam_policy.node_ebs_csi.arn
}

# ---------------------------------------------------------------------------
# IAM — a second principal to grant read-only cluster access to
# ---------------------------------------------------------------------------

resource "aws_iam_role" "viewer" {
  name = "${var.cluster_name}-viewer"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect    = "Allow"
      Action    = "sts:AssumeRole"
      Principal = { Service = "ec2.amazonaws.com" }
    }]
  })
}

# ---------------------------------------------------------------------------
# ECR — repository the workers pull the demo image from
# ---------------------------------------------------------------------------

resource "aws_ecr_repository" "demo" {
  name = "spinifex-demo"

  # tofu destroy must remove the repo even though it still holds the pushed demo
  # image; without this, DeleteRepository (force=false) fails RepositoryNotEmpty.
  force_delete = true
}

# ---------------------------------------------------------------------------
# EKS cluster — public + private endpoints, LBC-owned ingress
#
# The spinifex.io/managed-ingress = "false" tag disables K3s' built-in
# traefik/servicelb so the AWS Load Balancer Controller owns ingress, matching
# how ingress works on AWS EKS.
# ---------------------------------------------------------------------------

resource "aws_eks_cluster" "this" {
  name     = var.cluster_name
  role_arn = aws_iam_role.cluster.arn
  version  = var.k8s_version

  vpc_config {
    subnet_ids              = [aws_subnet.private_a.id, aws_subnet.private_b.id]
    endpoint_public_access  = true
    endpoint_private_access = true
    public_access_cidrs     = [var.api_public_access_cidr]
  }

  access_config {
    authentication_mode                         = "API"
    bootstrap_cluster_creator_admin_permissions = true
  }

  depends_on = [aws_iam_role_policy_attachment.cluster]

  # Spinifex's DescribeCluster doesn't echo these back the way the AWS provider
  # expects: access_config reads as absent (and its ForceNew
  # bootstrap_cluster_creator_admin_permissions would replace the live cluster on
  # re-add), and vpc_config.security_group_ids reads back as drift. Ignore both so
  # later applies (e.g. adding the Argo CD NodePort rule) neither replace the
  # cluster nor detach its security groups.
  lifecycle {
    ignore_changes = [access_config, vpc_config[0].security_group_ids]
  }

  tags = {
    Name                          = var.cluster_name
    "spinifex.io/managed-ingress" = "false"
  }
}

# ---------------------------------------------------------------------------
# Managed node group — workers in the private subnets
# ---------------------------------------------------------------------------

resource "aws_eks_node_group" "workers" {
  cluster_name    = aws_eks_cluster.this.name
  node_group_name = "workers"
  node_role_arn   = aws_iam_role.node.arn
  subnet_ids      = [aws_subnet.private_a.id, aws_subnet.private_b.id]

  scaling_config {
    desired_size = var.node_desired_size
    min_size     = var.node_desired_size
    max_size     = var.node_desired_size * 2
  }

  instance_types = [var.node_instance_type]
  ami_type       = "AL2_x86_64"

  depends_on = [
    aws_iam_role_policy_attachment.node_worker,
    aws_iam_role_policy_attachment.node_cni,
    aws_iam_role_policy_attachment.node_ecr,
    aws_iam_role_policy_attachment.node_lbc,
    aws_iam_role_policy_attachment.node_ebs_csi,
  ]

  tags = {
    Name = "${var.cluster_name}-workers"
  }
}

# ---------------------------------------------------------------------------
# Addon — AWS Load Balancer Controller
#
# Spinifex wires the controller's AWS credentials and ELB-eligible subnets at the
# node level, so no IRSA role or subnet tagging is needed here. addon_version is
# omitted: the AWS provider demands a v-prefixed version that the catalog rejects,
# so let Spinifex default to its catalog version.
# ---------------------------------------------------------------------------

resource "aws_eks_addon" "lbc" {
  cluster_name                = aws_eks_cluster.this.name
  addon_name                  = "aws-load-balancer-controller"
  resolve_conflicts_on_create = "OVERWRITE"

  depends_on = [aws_eks_node_group.workers]

  tags = {
    Name = "${var.cluster_name}-lbc"
  }
}

# ---------------------------------------------------------------------------
# Addon — Argo CD (GitOps delivery)
#
# Installed through the EKS API; Spinifex stages the bundle host-side and the
# worker renders it into the K3s auto-deploy dir. The workloads module registers
# the git repo and creates the Argo CD Application that syncs the demo app.
# addon_version omitted (see the LBC note).
# ---------------------------------------------------------------------------

resource "aws_eks_addon" "argocd" {
  cluster_name                = aws_eks_cluster.this.name
  addon_name                  = "argocd"
  resolve_conflicts_on_create = "OVERWRITE"

  depends_on = [aws_eks_node_group.workers]

  tags = {
    Name = "${var.cluster_name}-argocd"
  }
}

# ---------------------------------------------------------------------------
# Addon — EBS CSI driver (Viperblock-backed PersistentVolumes)
#
# Ships a default gp3 StorageClass (provisioner ebs.csi.aws.com). A PVC in the
# demo app's git manifests dynamically provisions a Viperblock-backed EBS volume
# so the app's state survives pod restarts. addon_version omitted (see LBC note).
# ---------------------------------------------------------------------------

resource "aws_eks_addon" "ebs_csi" {
  cluster_name                = aws_eks_cluster.this.name
  addon_name                  = "aws-ebs-csi-driver"
  resolve_conflicts_on_create = "OVERWRITE"

  depends_on = [aws_eks_node_group.workers]

  tags = {
    Name = "${var.cluster_name}-ebs-csi"
  }
}

# ---------------------------------------------------------------------------
# Access entry — grant the viewer principal read-only access (API auth mode)
# ---------------------------------------------------------------------------

resource "aws_eks_access_entry" "viewer" {
  cluster_name  = aws_eks_cluster.this.name
  principal_arn = aws_iam_role.viewer.arn
  type          = "STANDARD"
}

resource "aws_eks_access_policy_association" "viewer" {
  cluster_name  = aws_eks_cluster.this.name
  principal_arn = aws_iam_role.viewer.arn
  policy_arn    = "arn:aws:eks::aws:cluster-access-policy/AmazonEKSViewPolicy"

  access_scope {
    type = "cluster"
  }

  depends_on = [aws_eks_access_entry.viewer]
}

# ---------------------------------------------------------------------------
# TLS — self-signed certificate imported into ACM
#
# Spinifex ACM supports ImportCertificate (not RequestCertificate), so the cert
# is generated locally by the tls provider and imported. The workloads Ingress
# attaches it to the ALB's HTTPS listener by ARN.
# ---------------------------------------------------------------------------

resource "tls_private_key" "ingress" {
  algorithm = "RSA"
  rsa_bits  = 2048
}

resource "tls_self_signed_cert" "ingress" {
  private_key_pem = tls_private_key.ingress.private_key_pem

  subject {
    common_name  = var.cert_common_name
    organization = "Spinifex EKS Demo"
  }

  # Wildcard SAN so one ALB (shared IngressGroup) can host-route both the demo
  # app (app.<cn>) and the Argo CD UI (argocd.<cn>) off this single cert.
  dns_names             = [var.cert_common_name, "*.${var.cert_common_name}"]
  validity_period_hours = 8760
  early_renewal_hours   = 720

  allowed_uses = [
    "key_encipherment",
    "digital_signature",
    "server_auth",
  ]
}

resource "aws_acm_certificate" "ingress" {
  private_key      = tls_private_key.ingress.private_key_pem
  certificate_body = tls_self_signed_cert.ingress.cert_pem

  tags = {
    Name = "${var.cluster_name}-ingress"
  }
}

# ---------------------------------------------------------------------------
# Let the ALB reach the workers' NodePort
#
# Spinifex auto-manages the nodegroup SG and admits only intra-cluster traffic.
# The LBC-provisioned ALB lives in this VPC, so admit the VPC CIDR on the
# NodePort. Look the SG up by its deterministic name and add one rule.
# ---------------------------------------------------------------------------

data "aws_security_group" "nodegroup" {
  filter {
    name   = "group-name"
    values = ["eks-cluster-${var.cluster_name}-nodegroup-sg"]
  }

  filter {
    name   = "vpc-id"
    values = [aws_vpc.main.id]
  }

  depends_on = [aws_eks_node_group.workers]
}

resource "aws_vpc_security_group_ingress_rule" "nodeport_from_vpc" {
  security_group_id = data.aws_security_group.nodegroup.id
  cidr_ipv4         = aws_vpc.main.cidr_block
  from_port         = var.node_port
  to_port           = var.node_port
  ip_protocol       = "tcp"

  tags = {
    Name = "${var.cluster_name}-alb-nodeport"
  }
}

# Same, for the Argo CD UI NodePort its ALB forwards to.
resource "aws_vpc_security_group_ingress_rule" "argocd_nodeport_from_vpc" {
  security_group_id = data.aws_security_group.nodegroup.id
  cidr_ipv4         = aws_vpc.main.cidr_block
  from_port         = var.argocd_node_port
  to_port           = var.argocd_node_port
  ip_protocol       = "tcp"

  tags = {
    Name = "${var.cluster_name}-argocd-nodeport"
  }
}

# ---------------------------------------------------------------------------
# Outputs
# ---------------------------------------------------------------------------

output "cluster_name" {
  value = aws_eks_cluster.this.name
}

output "region" {
  value = var.region
}

output "node_port" {
  value = var.node_port
}

output "argocd_node_port" {
  value = var.argocd_node_port
}

output "node_desired_size" {
  value = var.node_desired_size
}

output "certificate_arn" {
  value = aws_acm_certificate.ingress.arn
}

output "cert_common_name" {
  value = var.cert_common_name
}

output "viewer_principal_arn" {
  value = aws_iam_role.viewer.arn
}

output "ecr_repository_url" {
  value       = aws_ecr_repository.demo.repository_url
  description = "Push the demo-app image here, then apply the workloads module"
}

output "ingress_address_hint" {
  value = "The demo app and Argo CD UI share one ALB (LBC IngressGroup), host-routed: app.${var.cert_common_name} and argocd.${var.cert_common_name}. Get the address: kubectl get ingress spinifex-demo -o jsonpath='{.status.loadBalancer.ingress[0].hostname}{\"\\n\"}'."
}

output "argocd_ingress_hint" {
  value = "Same ALB as the app. Open https://argocd.${var.cert_common_name} (admin password: kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath='{.data.password}' | base64 -d). Resolve the host to the ALB address, or curl -k --resolve."
}

output "update_kubeconfig" {
  value = "aws eks update-kubeconfig --name ${aws_eks_cluster.this.name} --region ${var.region}"
}

Step 2. Deploy the Cluster + Addons

bash
export AWS_PROFILE=spinifex
tofu init
tofu apply        # add -var node_desired_size=3 for an HA-shaped cluster

Wait for the cluster to reach ACTIVE and for all three addons (aws-load-balancer-controller, argocd, aws-ebs-csi-driver) to report healthy:

bash
aws eks list-addons --cluster-name eks-gitops
aws eks describe-addon --cluster-name eks-gitops --addon-name argocd --query 'addon.status'

Step 3. Build and Push the Demo Image

bash
cd ../demo-app
REGISTRY=$(cd ../eks-gitops-argocd && tofu output -raw ecr_repository_url)
REGISTRY_HOST=${REGISTRY%%/*}
aws ecr get-login-password | docker login --username AWS --password-stdin "$REGISTRY_HOST"
docker build -t "${REGISTRY}:latest" .
docker push "${REGISTRY}:latest"
cd ../eks-gitops-argocd

Point the app's manifests at this image: set the image ref in eks-demo-app/manifests (a kustomize images: override) to your ${REGISTRY}:latest, and push that to the git repo.

Step 4. Hand Delivery to Argo CD

bash
cd workloads
tofu init
tofu apply \
  -var git_repo_url=https://github.com/mulgadc/eks-demo-app.git \
  -var git_token=<read-only-PAT>     # omit for a public repo

This registers the repo credential, creates the Argo CD Application, the demo app's HTTPS Ingress, and an HTTPS Ingress for the Argo CD UI itself. Argo CD then syncs the Deployment, Service, and PVC from git.

hcl
# GitOps workload for eks-gitops-argocd — separate root module / state.
#
# Instead of applying the app directly, this module hands delivery to Argo CD:
# it registers the (private) git repo as an Argo CD repository credential and
# creates an Argo CD Application that syncs the demo app's manifests from it. The
# app's Deployment, Service, and PersistentVolumeClaim live in the git repo (see
# ../../../../eks-demo-app); the HTTPS Ingress (LBC + ACM) stays here and points
# at the Service that Argo CD creates.
#
# The Argo CD Application is a kubernetes_manifest, so the argoproj.io CRDs must
# already exist — apply the parent module (which installs the argocd addon) and
# let it reach ACTIVE before applying this module.
#
# Usage:
#   cd spinifex/docs/terraform-workbooks/eks-gitops-argocd
#   tofu init && tofu apply                       # parent: cluster + addons + ACM
#   cd workloads && tofu init
#   tofu apply -var git_repo_url=https://github.com/mulgadc/eks-demo-app.git \
#              -var git_token=<a-read-only-PAT>
#   # teardown is the reverse — destroy here first, then the parent.

terraform {
  required_version = ">= 1.6.0"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 5.40, < 6.0"
    }
    kubernetes = {
      source  = "hashicorp/kubernetes"
      version = ">= 2.20"
    }
  }
}

variable "spinifex_endpoint" {
  type    = string
  default = "https://127.0.0.1:9999"
}

variable "git_repo_url" {
  type        = string
  default     = "https://github.com/mulgadc/eks-demo-app.git"
  description = "Git repo Argo CD syncs the demo app from"
}

variable "git_revision" {
  type        = string
  default     = "main"
  description = "Git branch, tag, or commit Argo CD tracks"
}

variable "git_path" {
  type        = string
  default     = "manifests"
  description = "Path within the repo holding the app manifests"
}

variable "git_username" {
  type        = string
  default     = "git"
  description = "Username for the git credential (any non-empty value for a PAT)"
}

variable "git_token" {
  type        = string
  default     = ""
  sensitive   = true
  description = "Read-only personal access token for the private repo. Leave empty for a public repo."
}

provider "aws" {
  region = data.terraform_remote_state.infra.outputs.region

  endpoints {
    ec2 = var.spinifex_endpoint
    iam = var.spinifex_endpoint
    sts = var.spinifex_endpoint
    eks = var.spinifex_endpoint
  }

  skip_credentials_validation = true
  skip_metadata_api_check     = true
  skip_requesting_account_id  = true
  skip_region_validation      = true
}

data "terraform_remote_state" "infra" {
  backend = "local"

  config = {
    path = "../terraform.tfstate"
  }
}

locals {
  cluster_name     = data.terraform_remote_state.infra.outputs.cluster_name
  region           = data.terraform_remote_state.infra.outputs.region
  cert_arn         = data.terraform_remote_state.infra.outputs.certificate_arn
  argocd_node_port = data.terraform_remote_state.infra.outputs.argocd_node_port
  cert_cn          = data.terraform_remote_state.infra.outputs.cert_common_name
  private_repo     = var.git_token != ""

  # The demo app and the Argo CD UI share one ALB via an LBC IngressGroup; LBC
  # gives each backend its own target group and host-routes between them.
  alb_group   = local.cluster_name
  app_host    = "app.${local.cert_cn}"
  argocd_host = "argocd.${local.cert_cn}"
}

data "aws_eks_cluster" "this" {
  name = local.cluster_name
}

provider "kubernetes" {
  host                   = data.aws_eks_cluster.this.endpoint
  cluster_ca_certificate = base64decode(data.aws_eks_cluster.this.certificate_authority[0].data)

  exec {
    api_version = "client.authentication.k8s.io/v1beta1"
    command     = "aws"
    args        = ["eks", "get-token", "--cluster-name", local.cluster_name, "--region", local.region]
  }
}

# Repository credential for the private repo. Argo CD picks up Secrets in its
# namespace labelled argocd.argoproj.io/secret-type=repository. Created only when
# a token is supplied (a public repo needs no credential).
resource "kubernetes_secret_v1" "repo" {
  count = local.private_repo ? 1 : 0

  metadata {
    name      = "eks-demo-app-repo"
    namespace = "argocd"
    labels = {
      "argocd.argoproj.io/secret-type" = "repository"
    }
  }

  data = {
    type     = "git"
    url      = var.git_repo_url
    username = var.git_username
    password = var.git_token
  }
}

# Argo CD Application: syncs the demo app from the git repo into the default
# namespace, self-healing and pruning so the cluster tracks the repo.
resource "kubernetes_manifest" "demo_app" {
  manifest = {
    apiVersion = "argoproj.io/v1alpha1"
    kind       = "Application"

    # The resources finalizer makes deleting this Application cascade-prune the
    # synced resources (Deployment, Service, PVC). Without it, tofu destroy drops
    # the Application CR but orphans the live PVC, whose Delete-reclaim PV then
    # leaks an EBS volume once the cluster (and ebs-csi) is torn down.
    metadata = {
      name       = "spinifex-demo"
      namespace  = "argocd"
      finalizers = ["resources-finalizer.argocd.argoproj.io"]
    }

    spec = {
      project = "default"

      source = {
        repoURL        = var.git_repo_url
        targetRevision = var.git_revision
        path           = var.git_path
      }

      destination = {
        server    = "https://kubernetes.default.svc"
        namespace = "default"
      }

      syncPolicy = {
        automated = {
          prune    = true
          selfHeal = true
        }
        syncOptions = ["CreateNamespace=true"]
      }
    }
  }

  depends_on = [kubernetes_secret_v1.repo]
}

# HTTPS Ingress (LBC + ACM), carried over from eks-https-ingress. It points at
# the spinifex-demo Service that Argo CD creates from the git manifests.
resource "kubernetes_ingress_v1" "demo" {
  metadata {
    name      = "spinifex-demo"
    namespace = "default"

    annotations = {
      "alb.ingress.kubernetes.io/scheme"           = "internet-facing"
      "alb.ingress.kubernetes.io/target-type"      = "instance"
      "alb.ingress.kubernetes.io/group.name"       = local.alb_group
      "alb.ingress.kubernetes.io/listen-ports"     = "[{\"HTTP\":80},{\"HTTPS\":443}]"
      "alb.ingress.kubernetes.io/certificate-arn"  = local.cert_arn
      "alb.ingress.kubernetes.io/ssl-redirect"     = "443"
      "alb.ingress.kubernetes.io/healthcheck-path" = "/healthz"
    }
  }

  spec {
    ingress_class_name = "alb"

    # No host condition: the app is the ALB's catch-all on :443, so it answers on
    # the raw ALB IP as well as app.<cn> once DNS is wired. Argo CD's host rule is
    # more specific and still wins for argocd.<cn>.
    rule {
      http {
        path {
          path      = "/"
          path_type = "Prefix"

          backend {
            service {
              name = "spinifex-demo"
              port {
                number = 80
              }
            }
          }
        }
      }
    }
  }

  depends_on = [kubernetes_manifest.demo_app]
}

# ---------------------------------------------------------------------------
# Argo CD UI exposure — managing the cluster through the GitOps console is the
# point of this demo, so the UI gets the same HTTPS-Ingress treatment as the app
# rather than a port-forward. The argocd addon ships argocd-server as a ClusterIP
# only; expose it via a NodePort the ALB targets. argocd-server serves TLS on
# 8080 (no --insecure), so the ALB speaks HTTPS to the backend.
#
# Same group.name as the demo Ingress, so LBC folds both onto ONE ALB. On :443
# the app is the catch-all and argocd.<cn> host-routes to the UI. Argo CD also
# gets a hostless :8443 listener (argocd_ip below) so the UI is reachable on the
# raw ALB IP before DNS exists — https://<alb-ip>:8443.
# ---------------------------------------------------------------------------

resource "kubernetes_service_v1" "argocd_server_nodeport" {
  metadata {
    name      = "argocd-server-nodeport"
    namespace = "argocd"
  }

  spec {
    type     = "NodePort"
    selector = { "app.kubernetes.io/name" = "argocd-server" }

    port {
      port        = 443
      target_port = 8080
      node_port   = local.argocd_node_port
      protocol    = "TCP"
    }
  }
}

resource "kubernetes_ingress_v1" "argocd" {
  metadata {
    name      = "argocd-server"
    namespace = "argocd"

    annotations = {
      "alb.ingress.kubernetes.io/scheme"               = "internet-facing"
      "alb.ingress.kubernetes.io/target-type"          = "instance"
      "alb.ingress.kubernetes.io/group.name"           = local.alb_group
      "alb.ingress.kubernetes.io/listen-ports"         = "[{\"HTTPS\":443}]"
      "alb.ingress.kubernetes.io/certificate-arn"      = local.cert_arn
      "alb.ingress.kubernetes.io/backend-protocol"     = "HTTPS"
      "alb.ingress.kubernetes.io/healthcheck-protocol" = "HTTPS"
      "alb.ingress.kubernetes.io/healthcheck-path"     = "/healthz"
    }
  }

  spec {
    ingress_class_name = "alb"

    rule {
      host = local.argocd_host

      http {
        path {
          path      = "/"
          path_type = "Prefix"

          backend {
            service {
              name = kubernetes_service_v1.argocd_server_nodeport.metadata[0].name
              port {
                number = 443
              }
            }
          }
        }
      }
    }
  }
}

# Hostless Argo CD listener on :8443 of the same ALB, so the UI is reachable on
# the raw ALB IP (https://<alb-ip>:8443) without DNS or a host header. argocd.<cn>
# on :443 still works via the host-routed ingress above.
resource "kubernetes_ingress_v1" "argocd_ip" {
  metadata {
    name      = "argocd-server-ip"
    namespace = "argocd"

    annotations = {
      "alb.ingress.kubernetes.io/scheme"               = "internet-facing"
      "alb.ingress.kubernetes.io/target-type"          = "instance"
      "alb.ingress.kubernetes.io/group.name"           = local.alb_group
      "alb.ingress.kubernetes.io/listen-ports"         = "[{\"HTTPS\":8443}]"
      "alb.ingress.kubernetes.io/certificate-arn"      = local.cert_arn
      "alb.ingress.kubernetes.io/backend-protocol"     = "HTTPS"
      "alb.ingress.kubernetes.io/healthcheck-protocol" = "HTTPS"
      "alb.ingress.kubernetes.io/healthcheck-path"     = "/healthz"
    }
  }

  spec {
    ingress_class_name = "alb"

    rule {
      http {
        path {
          path      = "/"
          path_type = "Prefix"

          backend {
            service {
              name = kubernetes_service_v1.argocd_server_nodeport.metadata[0].name
              port {
                number = 443
              }
            }
          }
        }
      }
    }
  }
}

output "application_name" {
  value = kubernetes_manifest.demo_app.manifest.metadata.name
}

output "alb_address_hint" {
  value = "One shared ALB. Get its IP: kubectl get ingress spinifex-demo -o jsonpath='{.status.loadBalancer.ingress[0].ip}{\"\\n\"}' (or [0].hostname). Reach by raw IP — app: https://<alb-ip>/  Argo CD: https://<alb-ip>:8443/ . By DNS — app: ${local.app_host}  Argo CD: ${local.argocd_host}."
}

output "argocd_admin_password_cmd" {
  value = "kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath='{.data.password}' | base64 -d ; echo"
}

output "argocd_url_hint" {
  value = "Open https://${local.argocd_host} (admin password: kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath='{.data.password}' | base64 -d). Resolve the host to the ALB address (northstar/CNAME), or curl -k --resolve ${local.argocd_host}:443:<alb-ip>."
}

Step 5. Watch the Sync and Open the App

The demo app and the Argo CD UI share one ALB (an LBC IngressGroup). On :443 the app is the catch-all and argocd.eks-gitops.spinifex.local host-routes to the UI; Argo CD also gets a hostless :8443 listener so it is reachable on the raw ALB IP before DNS exists. Grab the shared ALB address once:

bash
# Point kubectl at the cluster first (writes ~/.kube/config) — without this every
# kubectl call fails to connect:
aws eks update-kubeconfig --name eks-gitops --region ap-southeast-2

kubectl -n argocd get applications
kubectl get pods -o wide
kubectl get pvc                    # the EBS-CSI volume should be Bound

DNSNAME=$(kubectl get ingress spinifex-demo -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
ALB_IP=$(aws elbv2 describe-load-balancers \
  --query "LoadBalancers[?DNSName=='${DNSNAME}'].AvailabilityZones[].LoadBalancerAddresses[].IpAddress | [0]" \
  --output text)
echo "$ALB_IP"

Reach it by raw IP (no DNS, no host header):

ServiceURL
Demo apphttps://<ALB_IP>/
Argo CD UIhttps://<ALB_IP>:8443/
bash
curl -k https://"$ALB_IP"/            # app — returns the Spinifex page
curl -k https://"$ALB_IP":8443/       # Argo CD UI

The bare ALB IP serves the app on :443. Hitting https://<ALB_IP>/ and expecting Argo CD gives the app instead — use :8443 for the UI. Once northstar (or Route 53) resolves the hostnames to the ALB, app.eks-gitops.spinifex.local and argocd.eks-gitops.spinifex.local both work on :443. Self-signed cert — accept the browser warning.

The page shows the "persisted to EBS volume" badge and a hit counter that keeps climbing — delete the pod (kubectl delete pod -l app=spinifex-demo) and the count survives, proving the volume is durable.

Step 6. Open the Argo CD UI

Managing deployments through the Argo CD console is the point of this workbook, so the UI is exposed on the same ALB as the app, not behind a port-forward. The workloads module adds a NodePort Service in front of argocd-server (the addon ships it ClusterIP only) and two Ingresses in the shared group — a host-routed one on :443 and a hostless one on :8443 for raw-IP access. argocd-server serves TLS on its own port, so both use backend-protocol: HTTPS.

Get the admin credentials. Argo CD generates a one-time admin password into a Secret on install; the username is always admin:

bash
kubectl -n argocd get secret argocd-initial-admin-secret \
  -o jsonpath='{.data.password}' | base64 -d ; echo

(tofu output argocd_admin_password_cmd prints this same command.)

Open the UI and log in as admin:

  • By raw IP: https://<ALB_IP>:8443/
  • By DNS once wired: https://argocd.eks-gitops.spinifex.local

The spinifex-demo Application shows the sync status, the resource tree, and live diffs against git — change a manifest in the repo and watch Argo CD reconcile it.

If you'd rather not expose the UI at all, port-forward instead:

bash
kubectl -n argocd port-forward svc/argocd-server 8080:443
# then open https://localhost:8080

Cleanup

Destroy in reverse — the Argo CD Application + Ingress first (so Argo CD prunes the workload and the LBC tears the ALB down), then the cluster:

bash
cd workloads
tofu destroy
cd ..
tofu destroy

Troubleshooting

Application Won't Sync

bash
kubectl -n argocd get applications
kubectl -n argocd describe application spinifex-demo     # conditions show repo/auth errors
kubectl -n argocd logs deploy/argocd-repo-server --tail=50

For a private repo, confirm the eks-demo-app-repo Secret exists in the argocd namespace with the argocd.argoproj.io/secret-type=repository label and a valid token.

kubernetes_manifest Fails: no matches for kind "Application"

The argoproj.io CRDs aren't installed yet. Apply the parent module and wait for the argocd addon to reach ACTIVE before applying workloads/.

PVC Stuck in Pending

bash
kubectl get pvc
kubectl describe pvc <name>
kubectl get storageclass

Confirm the aws-ebs-csi-driver addon is ACTIVE and the default StorageClass exists. The app still serves with an in-memory counter if the volume never binds.

The Ingress Has No Address

The LBC populates status.loadBalancer once it provisions the ALB. Check the addon and the controller logs:

bash
kubectl describe ingress spinifex-demo
kubectl -n kube-system logs deploy/aws-load-balancer-controller --tail=50

Confirm the cluster carries spinifex.io/managed-ingress = "false" so the LBC owns ingress.

If the controller logs show FailedBuildModel ... DescribeAvailabilityZones ... 403 ... AccessDenied, the node role is missing the LBC permissions. The controller runs with the node instance-profile credentials, so the ${var.cluster_name}-node-lbc policy (aws_iam_policy.node_lbc) must be attached to the node role — re-run tofu apply on the parent module if it was provisioned before that policy existed.

Provider Connection Refused

bash
sudo systemctl status spinifex.target
curl -k https://localhost:9999/