Skip to content
Mulga mulga

S3-Backed Web App

Deploy a Flask file-sharing web application backed by S3 (Predastore) using Terraform on Spinifex.

terraforms3predastoreflaskwebappworkbook

Overview

Deploy an EC2 instance running a Flask file-sharing web application backed by S3 (Predastore). Users can upload files through a web form and browse uploaded content — demonstrating Terraform managing both compute and object storage together.

Architecture:

S3 webapp — browser to Flask EC2 instance to Predastore via S3 API

What you'll learn:

  • Configuring the AWS provider with both Spinifex and Predastore endpoints
  • Creating S3 buckets on Predastore via Terraform
  • Deploying a Python webapp with cloud-init that talks to S3
  • Passing credentials and configuration to instances via user-data

Prerequisites:

  • Spinifex installed and running (see Installing Spinifex)
  • Predastore running (S3 API on port 8443)
  • A Debian 12 AMI imported (see Setting Up Your Cluster)
  • OpenTofu or Terraform installed
  • The EC2 instance must be able to reach Predastore — use the host's br-wan IP, not localhost

Instructions

Step 1. Get the Template

Clone the Terraform examples from the Spinifex repository:

bash
git clone --depth 1 --filter=blob:none --sparse https://github.com/mulgadc/spinifex.git spinifex-tf
cd spinifex-tf
git sparse-checkout set docs/terraform
cd docs/terraform/s3-webapp

Or create the files manually and paste the full configuration below.

Step 2. Create terraform.tfvars

Before deploying, create a terraform.tfvars with your Predastore credentials. The predastore_host must be reachable from inside the VPC — use the host's br-wan or LAN IP, not localhost.

hcl
# Copy this to terraform.tfvars and fill in your values.
#
# The predastore_host must be reachable from INSIDE the VPC guest — not
# localhost. Use the host's br-wan or LAN IP, e.g. "192.168.1.10:8443".

predastore_host = "192.168.1.10:8443"
s3_access_key   = "AKIAIOSFODNN7EXAMPLE"
s3_secret_key   = "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"

# Optional overrides:
# spinifex_endpoint    = "https://localhost:9999"
# predastore_endpoint  = "https://localhost:8443"
# bucket_name          = "webapp-uploads"
# region               = "ap-southeast-2"

Step 3. Create main.tf

hcl
# Example 3: S3-Backed Web Application
#
# Deploys an EC2 instance running a simple file-sharing webapp backed by S3
# (Predastore). Users can upload files through a web form and browse uploaded
# content — demonstrating Terraform managing both compute (Spinifex) and
# object storage (Predastore) resources together.
#
# Architecture:
#
#   Browser ──HTTP──▶ EC2 Instance (Flask webapp, port 80)
#                         │
#                         ▼ S3 API (boto3)
#                     Predastore (port 8443)
#
# Prerequisites:
#   - Spinifex services running (gateway on port 9999)
#   - Predastore running (S3 API on port 8443)
#   - The EC2 instance must be able to reach the Predastore endpoint.
#     Set `predastore_host` to the IP reachable from inside the VPC
#     (e.g. the host's br-wan IP, NOT localhost).
#
# Usage:
#   cd spinifex/scripts/iac/aws/examples/03-s3-webapp
#   export AWS_PROFILE=spinifex
#   tofu init && tofu apply
#
# After apply:
#   curl http://<public_ip>          # File browser UI
#   ssh -i s3-webapp-demo.pem ec2-user@<public_ip>

terraform {
  required_version = ">= 1.6.0"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 5.0"
    }
    tls = {
      source  = "hashicorp/tls"
      version = ">= 4.0"
    }
    local = {
      source  = "hashicorp/local"
      version = ">= 2.0"
    }
  }
}

# ---------------------------------------------------------------------------
# Variables
# ---------------------------------------------------------------------------

variable "region" {
  type    = string
  default = "ap-southeast-2"
}

variable "spinifex_endpoint" {
  type        = string
  default     = "https://localhost:9999"
  description = "Spinifex AWS gateway endpoint (EC2/IAM)"
}

variable "predastore_endpoint" {
  type        = string
  default     = "https://localhost:8443"
  description = "Predastore S3 endpoint (for Terraform to create buckets)"
}

variable "predastore_host" {
  type        = string
  description = "Predastore host:port reachable from inside the VPC (e.g. 192.168.1.10:8443)"
}

variable "s3_access_key" {
  type        = string
  description = "S3 access key for Predastore"
}

variable "s3_secret_key" {
  type        = string
  sensitive   = true
  description = "S3 secret key for Predastore"
}

variable "bucket_name" {
  type    = string
  default = "webapp-uploads"
}

# ---------------------------------------------------------------------------
# Provider — EC2 via Spinifex gateway, S3 via Predastore
# ---------------------------------------------------------------------------

provider "aws" {
  region     = var.region
  access_key = var.s3_access_key
  secret_key = var.s3_secret_key

  endpoints {
    ec2 = var.spinifex_endpoint
    s3  = var.predastore_endpoint
    iam = var.spinifex_endpoint
    sts = var.spinifex_endpoint
  }

  s3_use_path_style = true

  skip_credentials_validation = true
  skip_metadata_api_check     = true
  skip_requesting_account_id  = true
  skip_region_validation      = true
}

# ---------------------------------------------------------------------------
# Data sources
# ---------------------------------------------------------------------------

data "aws_availability_zones" "available" {
  state = "available"
}

data "aws_ami" "debian12" {
  most_recent = true
  owners      = ["000000000000"]

  filter {
    name   = "name"
    values = ["*debian-12*"]
  }

  filter {
    name   = "virtualization-type"
    values = ["hvm"]
  }

  filter {
    name   = "root-device-type"
    values = ["ebs"]
  }
}

# ---------------------------------------------------------------------------
# S3 Bucket (created on Predastore)
# ---------------------------------------------------------------------------

resource "aws_s3_bucket" "uploads" {
  bucket = var.bucket_name
}

# ---------------------------------------------------------------------------
# SSH Key Pair
# ---------------------------------------------------------------------------

resource "tls_private_key" "webapp" {
  algorithm = "ED25519"
}

resource "aws_key_pair" "webapp" {
  key_name   = "s3-webapp-demo"
  public_key = tls_private_key.webapp.public_key_openssh
}

resource "local_file" "webapp_pem" {
  filename        = "${path.module}/s3-webapp-demo.pem"
  content         = tls_private_key.webapp.private_key_openssh
  file_permission = "0600"
}

# ---------------------------------------------------------------------------
# VPC + Public Subnet
# ---------------------------------------------------------------------------

resource "aws_vpc" "main" {
  cidr_block           = "10.30.0.0/16"
  enable_dns_hostnames = true
  enable_dns_support   = true

  tags = {
    Name = "s3-webapp-demo-vpc"
  }
}

resource "aws_internet_gateway" "igw" {
  vpc_id = aws_vpc.main.id

  tags = {
    Name = "s3-webapp-demo-igw"
  }
}

resource "aws_subnet" "public" {
  vpc_id                  = aws_vpc.main.id
  cidr_block              = "10.30.1.0/24"
  availability_zone       = data.aws_availability_zones.available.names[0]
  map_public_ip_on_launch = true

  tags = {
    Name = "s3-webapp-demo-public"
  }
}

resource "aws_route_table" "public" {
  vpc_id = aws_vpc.main.id

  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = aws_internet_gateway.igw.id
  }

  tags = {
    Name = "s3-webapp-demo-public-rt"
  }
}

resource "aws_route_table_association" "public" {
  subnet_id      = aws_subnet.public.id
  route_table_id = aws_route_table.public.id
}

# ---------------------------------------------------------------------------
# Security Group — SSH + HTTP inbound, all outbound
# ---------------------------------------------------------------------------

resource "aws_security_group" "webapp" {
  name        = "s3-webapp-demo-sg"
  description = "Allow SSH and HTTP inbound"
  vpc_id      = aws_vpc.main.id

  ingress {
    description = "SSH"
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  ingress {
    description = "HTTP"
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  egress {
    description = "All outbound"
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags = {
    Name = "s3-webapp-demo-sg"
  }
}

# ---------------------------------------------------------------------------
# EC2 Instance — Flask webapp that talks to Predastore S3
# ---------------------------------------------------------------------------

resource "aws_instance" "webapp" {
  ami           = data.aws_ami.debian12.id
  instance_type = "t3.small"

  subnet_id              = aws_subnet.public.id
  vpc_security_group_ids = [aws_security_group.webapp.id]
  key_name               = aws_key_pair.webapp.key_name

  associate_public_ip_address = true

  user_data_base64 = base64encode(<<-USERDATA
    #!/bin/bash
    set -euo pipefail

    # Install dependencies
    apt-get update -y
    apt-get install -y python3-pip python3-venv

    # Create app directory and virtualenv
    mkdir -p /opt/webapp
    python3 -m venv /opt/webapp/venv
    /opt/webapp/venv/bin/pip install flask boto3

    # Write S3 credentials config
    cat > /opt/webapp/.env <<'ENVFILE'
    S3_ENDPOINT=https://${var.predastore_host}
    S3_BUCKET=${var.bucket_name}
    S3_ACCESS_KEY=${var.s3_access_key}
    S3_SECRET_KEY=${var.s3_secret_key}
    S3_REGION=${var.region}
    ENVFILE

    # Write the Flask application
    cat > /opt/webapp/app.py <<'PYEOF'
    import os, io, urllib3
    from flask import Flask, request, redirect, url_for, Response

    # Suppress TLS warnings for self-signed certs
    urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)

    # Load env
    env = {}
    with open("/opt/webapp/.env") as f:
        for line in f:
            line = line.strip()
            if "=" in line and not line.startswith("#"):
                k, v = line.split("=", 1)
                env[k] = v

    import boto3
    from botocore.config import Config

    s3 = boto3.client(
        "s3",
        endpoint_url=env["S3_ENDPOINT"],
        aws_access_key_id=env["S3_ACCESS_KEY"],
        aws_secret_access_key=env["S3_SECRET_KEY"],
        region_name=env["S3_REGION"],
        verify=False,
        config=Config(s3={"addressing_style": "path"}),
    )

    BUCKET = env["S3_BUCKET"]
    app = Flask(__name__)

    @app.route("/")
    def index():
        # List objects in the bucket
        try:
            resp = s3.list_objects_v2(Bucket=BUCKET)
            objects = resp.get("Contents", [])
        except Exception as e:
            objects = []

        rows = ""
        for obj in objects:
            key = obj["Key"]
            size = obj["Size"]
            rows += f'<tr><td><a href="/files/{key}">{key}</a></td><td>{size} bytes</td></tr>\n'

        return f"""<!DOCTYPE html>
    <html>
    <head><title>Spinifex S3 File Browser</title></head>
    <body style="font-family: sans-serif; max-width: 700px; margin: 40px auto;">
      <h1>Spinifex S3 File Browser</h1>
      <p>Bucket: <code>{BUCKET}</code></p>

      <h2>Upload a File</h2>
      <form method="POST" action="/upload" enctype="multipart/form-data">
        <input type="file" name="file" required>
        <button type="submit">Upload</button>
      </form>

      <h2>Files</h2>
      <table border="1" cellpadding="6" cellspacing="0" style="border-collapse: collapse;">
        <tr><th>Key</th><th>Size</th></tr>
        {rows if rows else '<tr><td colspan="2">No files yet</td></tr>'}
      </table>

      <hr>
      <p><small>Powered by Spinifex + Predastore</small></p>
    </body>
    </html>"""

    @app.route("/upload", methods=["POST"])
    def upload():
        f = request.files.get("file")
        if not f or not f.filename:
            return redirect("/")
        s3.put_object(Bucket=BUCKET, Key=f.filename, Body=f.read())
        return redirect("/")

    @app.route("/files/<path:key>")
    def download(key):
        try:
            obj = s3.get_object(Bucket=BUCKET, Key=key)
            return Response(
                obj["Body"].read(),
                headers={"Content-Disposition": f'inline; filename="{key}"'},
            )
        except Exception:
            return "Not found", 404

    if __name__ == "__main__":
        app.run(host="0.0.0.0", port=80)
    PYEOF

    # Create a systemd service so the webapp starts on boot
    cat > /etc/systemd/system/s3-webapp.service <<'SVCEOF'
    [Unit]
    Description=S3 File Browser Webapp
    After=network.target

    [Service]
    Type=simple
    ExecStart=/opt/webapp/venv/bin/python /opt/webapp/app.py
    WorkingDirectory=/opt/webapp
    Restart=always
    RestartSec=3

    [Install]
    WantedBy=multi-user.target
    SVCEOF

    systemctl daemon-reload
    systemctl enable s3-webapp
    systemctl start s3-webapp
  USERDATA
  )

  tags = {
    Name = "s3-webapp-demo"
  }
}

# ---------------------------------------------------------------------------
# Outputs
# ---------------------------------------------------------------------------

output "note" {
  value = "EC2 instances can take 30+ seconds to boot after apply. If SSH or HTTP is unreachable, wait and retry."
}

output "instance_id" {
  value = aws_instance.webapp.id
}

output "public_ip" {
  value = aws_instance.webapp.public_ip
}

output "bucket_name" {
  value = aws_s3_bucket.uploads.id
}

output "ssh_command" {
  value = "ssh -i s3-webapp-demo.pem ec2-user@${aws_instance.webapp.public_ip}"
}

output "web_url" {
  value = "http://${aws_instance.webapp.public_ip}"
}

Step 4. Deploy

bash
export AWS_PROFILE=spinifex
tofu init
tofu apply

Step 5. Test the Application

> Note: EC2 instances can take 30+ seconds to boot after apply. If SSH or HTTP is unreachable, wait and retry.

Open the web_url output in your browser. You should see the file browser UI. Upload a file and verify it appears in the list.

bash
# Verify via CLI
curl http://<public_ip>

# Check the S3 bucket directly
aws s3 ls s3://webapp-uploads/ --profile spinifex --endpoint-url https://localhost:8443

Clean Up

bash
tofu destroy

Troubleshooting

Predastore Connection Refused from Instance

The EC2 instance cannot reach localhost on the host. Set predastore_host to the host's br-wan or LAN IP address:

hcl
predastore_host = "192.168.1.10:8443"

S3 Bucket Creation Fails

Verify Predastore is running and accessible:

bash
curl -k https://localhost:8443/
aws s3 ls --profile spinifex --endpoint-url https://localhost:8443

Flask App Not Starting

SSH into the instance and check the service:

bash
ssh -i s3-webapp-demo.pem ec2-user@<public_ip>
sudo systemctl status s3-webapp
sudo journalctl -u s3-webapp --no-pager -n 50

Upload Fails or Files Not Appearing

Check that the S3 credentials in /opt/webapp/.env are correct and that the bucket exists:

bash
ssh -i s3-webapp-demo.pem ec2-user@<public_ip>
cat /opt/webapp/.env
/opt/webapp/venv/bin/python -c "import boto3; print('boto3 OK')"

AMI Not Found

Ensure you have imported a Debian 12 image:

bash
aws ec2 describe-images --owners 000000000000 --profile spinifex