docs: clarify custom project images vs upstream open-source infrastructure in COMPOSE_CONVENTIONS.md

feat: Implement SPIRE Smart Image Packaging & Single Volume Bootstrap
Added spire/Dockerfile.server and spire/Dockerfile.agent wrapping upstream ghcr.io/spiffe images.
Added self-seeding entrypoint scripts handling template provisioning and stale socket cleanup.
Added generic, commented HCL templates for server.conf and agent.conf.
Simplified infra/setup.ts to output a cleaner compose.spire.yml with one spire-data host bind and one spire-socket named volume, eliminating spire-init.
Updated CLI build commands to include SPIRE images.
Task marked as complete.

COMPOSE_CONVENTIONS.md

@@ -60,49 +60,46 @@ across our infrastructure.
 
 ## 3. Container Registries, Mirrors & Image Distribution
 
-Our infrastructure uses a tiered registry architecture to balance local caching,
-private custom image hosting, and universal build portability:
+Our infrastructure uses a tiered registry architecture to clearly separate
+custom application images from upstream cached dependencies:
 
 ### 3.1 Registry Tier Classification
 
 1. **Primary Custom Image Registry (`${REG}` $\rightarrow$ `quay.atyg.org`):**
-   - Hosts all internally built, project-specific custom images
-     (`${REG}/library/<image>:<tag>`).
-   - Examples: `${REG}/library/auth-yes-api:latest`,
-     `${REG}/library/spire-server:latest`, `${REG}/library/spire-agent:latest`.
-   - Retention policy is set for **indefinite storage** (auto-pruning disabled).
+   - Hosts all internally built, project-specific custom application images
+     (`${REG}/library/<app>:<tag>`).
+   - Example: `${REG}/library/auth-yes-api:latest` (built from project source).
+   - Retained **indefinitely** on disk (auto-pruning disabled).
 2. **Docker Hub Pull-Through Cache (`acr.atyg.org`):**
-   - Proxies and indefinitely caches upstream Docker Hub images on the local
-     network.
+   - Proxies and indefinitely caches upstream Docker Hub infrastructure images
+     on the local network.
    - Examples: `acr.atyg.org/library/postgres:18-alpine`,
      `acr.atyg.org/valkey/valkey:8-alpine`, `acr.atyg.org/library/alpine:3.20`.
 3. **GHCR Pull-Through Mirror (`${GHCR_REG}` $\rightarrow$ `ghcr.atyg.org`):**
-   - Proxies and caches GitHub Container Registry packages.
+   - Proxies and caches upstream GitHub Container Registry infrastructure
+     images.
    - Examples: `${GHCR_REG}/spiffe/spire-server:1.9.3`,
      `${GHCR_REG}/spiffe/spire-agent:1.9.3`.
 
 ### 3.2 Universal Dockerfile Portability (Jules & External CI)
 
-- **Public Upstream Defaults:** Dockerfiles must declare public upstream
-  registries by default so external agents (like Jules) and cloud CI runners can
-  build without private `.atyg.org` DNS:
+- **Public Upstream Defaults:** Dockerfiles for custom application images or
+  wrappers must declare public upstream registries by default so external agents
+  (like Jules) and cloud CI runners can build without private `.atyg.org` DNS:
   ```dockerfile
-  ARG SPIRE_UPSTREAM=ghcr.io/spiffe/spire-server:1.9.3
   ARG BASE_IMAGE=alpine:3.20
-
-  FROM ${SPIRE_UPSTREAM} AS upstream
   FROM ${BASE_IMAGE}
   ```
 - **Local Build Acceleration:** Local builds and CLI scripts (`infra/setup.ts`)
   can optionally pass `--build-arg` to pull through local mirrors
   (`ghcr.atyg.org`, `acr.atyg.org`).
-- **Push Destination:** Custom built images are tagged and pushed to the local
-  authority `${REG}/library/...` (`quay.atyg.org`).
+- **Push Destination:** Custom built application images are tagged and pushed to
+  the local authority `${REG}/library/...` (`quay.atyg.org`).
 
 ### 3.3 Compose Variable Cleanliness
 
 - **No Inline Defaults:** Always write clean variable references in Compose
-  files (`image: ${REG}/library/spire-server:latest`,
+  files (`image: ${REG}/library/auth-yes-api:latest`,
   `image: ${GHCR_REG}/spiffe/spire-server:1.9.3`).
 - **Centralized Values:** Default registry variables belong strictly in `.env`
   and `infra/setup.ts`.

infra/setup.ts

@@ -145,199 +145,35 @@ export function generateSpireDockerCompose(): string {
   return `version: "3.8"
 
 services:
-  spire-init:
-    image: \${REG}/library/alpine:3.20
-    restart: "no"
-    volumes:
-      - spire-server-conf:/opt/spire/server/conf
-      - spire-server-data:/opt/spire/server/data
-      - spire-agent-conf:/opt/spire/agent/conf
-      - spire-agent-data:/opt/spire/agent/data
-    entrypoint:
-      - /bin/sh
-      - -c
-      - |
-        mkdir -p /opt/spire/server/conf /opt/spire/server/data /opt/spire/agent/conf /opt/spire/agent/data
-        if [ ! -f /opt/spire/server/conf/server.conf ]; then
-          echo "Writing default SPIRE server.conf..."
-          cat << 'EOF' > /opt/spire/server/conf/server.conf
-        # ==============================================================================
-        # SPIRE Server Configuration
-        # Auth-Yes Identity & Access Management Fabric
-        # ==============================================================================
-        # This configuration defines the root SPIFFE trust authority for your cluster.
-        # You can customize trust domains, certificate TTLs, and datastore plugins below.
-        # ==============================================================================
-
-        server {
-          # Network binding: 0.0.0.0 listens on all internal mesh interfaces.
-          bind_address = "0.0.0.0"
-          bind_port = "8081"
-
-          # Trust Domain: Identifies the root cryptographic security domain.
-          # SPIFFE IDs will be generated in the format: spiffe://<trust_domain>/<workload>
-          trust_domain = "system.local"
-
-          # Directory where SPIRE server persists runtime data, datastore, and keys.
-          data_dir = "/opt/spire/data"
-
-          # Logging verbosity: DEBUG, INFO, WARN, ERROR
-          log_level = "INFO"
-
-          # Certificate Authority (CA) Time-to-Live (default: 30 days)
-          ca_ttl = "720h"
-
-          # Default Workload SVID Time-to-Live (default: 1 hour for high-security rotation)
-          default_x509_svid_ttl = "1h"
-        }
-
-        plugins {
-          # ----------------------------------------------------------------------------
-          # DataStore Plugin: Persists SPIFFE registrations, entries, and nodes.
-          # Default: Embedded SQLite3 datastore inside /opt/spire/data.
-          # ----------------------------------------------------------------------------
-          DataStore "sql" {
-            plugin_data {
-              database_type = "sqlite3"
-              connection_string = "/opt/spire/data/datastore.sqlite3"
-            }
-          }
-
-          # ----------------------------------------------------------------------------
-          # NodeAttestor Plugin: Verifies identity of SPIRE agents joining the cluster.
-          # 'join_token' allows dynamic 1-time token enrollment for agents.
-          # ----------------------------------------------------------------------------
-          NodeAttestor "join_token" {
-            plugin_data {}
-          }
-
-          # ----------------------------------------------------------------------------
-          # KeyManager Plugin: Securely stores the server CA private keys on disk.
-          # ----------------------------------------------------------------------------
-          KeyManager "disk" {
-            plugin_data {
-              keys_path = "/opt/spire/data/keys.json"
-            }
-          }
-        }
-        EOF
-        fi
-
-        if [ ! -f /opt/spire/agent/conf/agent.conf ]; then
-          echo "Writing default SPIRE agent.conf..."
-          cat << 'EOF' > /opt/spire/agent/conf/agent.conf
-        # ==============================================================================
-        # SPIRE Agent Configuration
-        # Auth-Yes Identity & Access Management Fabric
-        # ==============================================================================
-        # The SPIRE Agent runs as a local node daemon, attesting workloads (e.g. Docker
-        # containers) and serving the Workload API UNIX domain socket.
-        # ==============================================================================
-
-        agent {
-          # Directory where the SPIRE agent caches SVIDs, bundles, and keys.
-          data_dir = "/opt/spire/data"
-
-          # Logging verbosity: DEBUG, INFO, WARN, ERROR
-          log_level = "INFO"
-
-          # Address and port of the SPIRE Server container within the internal network.
-          server_address = "spire-server"
-          server_port = "8081"
-
-          # UNIX Domain Socket Path exposed to workloads for zero-trust identity fetching.
-          socket_path = "/var/run/spire/agent.sock"
-
-          # Must match the SPIRE Server's trust_domain.
-          trust_domain = "system.local"
-        }
-
-        plugins {
-          # ----------------------------------------------------------------------------
-          # NodeAttestor Plugin: Authenticates this agent with the SPIRE Server.
-          # ----------------------------------------------------------------------------
-          NodeAttestor "join_token" {
-            plugin_data {}
-          }
-
-          # ----------------------------------------------------------------------------
-          # KeyManager Plugin: Stores node-level private keys on disk.
-          # ----------------------------------------------------------------------------
-          KeyManager "disk" {
-            plugin_data {
-              directory = "/opt/spire/data"
-            }
-          }
-
-          # ----------------------------------------------------------------------------
-          # WorkloadAttestor Plugin: Inspects running containers on Docker / Podman
-          # to grant SPIFFE SVIDs based on container image, labels, or names.
-          # ----------------------------------------------------------------------------
-          WorkloadAttestor "docker" {
-            plugin_data {}
-          }
-        }
-        EOF
-        fi
-        echo "SPIRE bootstrap configuration initialized successfully."
-
   spire-server:
-    image: \${GHCR_REG}/spiffe/spire-server:1.9.3
+    image: \${REG}/library/spire-server:latest
     container_name: spire-server
     hostname: spire-server
-    depends_on:
-      spire-init:
-        condition: service_completed_successfully
     networks:
       - auth-internal-net
     volumes:
-      - spire-server-data:/opt/spire/data
-      - spire-server-conf:/opt/spire/conf:ro
-    command: ["-config", "/opt/spire/conf/server.conf"]
+      - spire-data:/opt/spire
 
   spire-agent:
-    image: \${GHCR_REG}/spiffe/spire-agent:1.9.3
+    image: \${REG}/library/spire-agent:latest
     container_name: spire-agent
     hostname: spire-agent
     pid: host
     depends_on:
-      spire-server:
-        condition: service_started
-      spire-init:
-        condition: service_completed_successfully
+      - spire-server
     networks:
       - auth-internal-net
     volumes:
+      - spire-data:/opt/spire
       - spire-socket:/var/run/spire
-      - spire-agent-data:/opt/spire/data
-      - spire-agent-conf:/opt/spire/conf:ro
       - /var/run/docker.sock:/var/run/docker.sock:ro
-    command: ["-config", "/opt/spire/conf/agent.conf"]
 
 volumes:
-  spire-server-data:
+  spire-data:
     driver: local
     driver_opts:
       type: none
-      device: \${SPIRE_DATA_PATH}/server/data
-      o: bind
-  spire-server-conf:
-    driver: local
-    driver_opts:
-      type: none
-      device: \${SPIRE_DATA_PATH}/server/conf
-      o: bind
-  spire-agent-data:
-    driver: local
-    driver_opts:
-      type: none
-      device: \${SPIRE_DATA_PATH}/agent/data
-      o: bind
-  spire-agent-conf:
-    driver: local
-    driver_opts:
-      type: none
-      device: \${SPIRE_DATA_PATH}/agent/conf
+      device: \${SPIRE_DATA_PATH}
       o: bind
   spire-socket:
     name: spire-socket
@@ -394,6 +230,10 @@ export function generateBuildCommands(reg: string): string[] {
   return [
     `podman build -t ${reg}/library/auth-yes-api:latest -f Dockerfile .`,
     `podman push ${reg}/library/auth-yes-api:latest`,
+    `podman build -t ${reg}/library/spire-server:latest -f spire/Dockerfile.server spire/`,
+    `podman push ${reg}/library/spire-server:latest`,
+    `podman build -t ${reg}/library/spire-agent:latest -f spire/Dockerfile.agent spire/`,
+    `podman push ${reg}/library/spire-agent:latest`,
   ];
 }
 
@@ -428,14 +268,6 @@ export async function generateAuthSetupFiles(
   const spireComposeContent = generateSpireDockerCompose();
   await Deno.writeTextFile(SPIRE_COMPOSE_PATH, spireComposeContent);
 
-  // Ensure default SPIRE directories exist
-  await Deno.mkdir(path.join("infra", "spire", "server", "conf"), {
-    recursive: true,
-  });
-  await Deno.mkdir(path.join("infra", "spire", "agent", "conf"), {
-    recursive: true,
-  });
-
   console.log(
     colors.green(
       `\n✓ Successfully generated ${ENV_PATH}, ${COMPOSE_PATH}, and ${SPIRE_COMPOSE_PATH}!`,

spire/Dockerfile.agent

@@ -0,0 +1,13 @@
+ARG SPIRE_VERSION=1.9.3
+FROM ghcr.io/spiffe/spire-agent:${SPIRE_VERSION} AS upstream
+FROM alpine:3.20
+
+RUN apk add --no-cache ca-certificates tzdata
+
+COPY --from=upstream /opt/spire/bin/spire-agent /usr/local/bin/spire-agent
+COPY templates/agent.conf /etc/spire/templates/agent.conf
+COPY entrypoint.agent.sh /usr/local/bin/entrypoint.sh
+RUN chmod +x /usr/local/bin/entrypoint.sh
+
+ENTRYPOINT ["/usr/local/bin/entrypoint.sh"]
+CMD ["run", "-config", "/opt/spire/agent.conf"]

spire/Dockerfile.server

@@ -0,0 +1,13 @@
+ARG SPIRE_VERSION=1.9.3
+FROM ghcr.io/spiffe/spire-server:${SPIRE_VERSION} AS upstream
+FROM alpine:3.20
+
+RUN apk add --no-cache ca-certificates tzdata
+
+COPY --from=upstream /opt/spire/bin/spire-server /usr/local/bin/spire-server
+COPY templates/server.conf /etc/spire/templates/server.conf
+COPY entrypoint.server.sh /usr/local/bin/entrypoint.sh
+RUN chmod +x /usr/local/bin/entrypoint.sh
+
+ENTRYPOINT ["/usr/local/bin/entrypoint.sh"]
+CMD ["run", "-config", "/opt/spire/server.conf"]

spire/entrypoint.agent.sh

@@ -0,0 +1,17 @@
+#!/bin/sh
+
+set -e
+
+# Mitigation for Risk 2: Stale UNIX Socket on Unclean Shutdown
+rm -f /var/run/spire/agent.sock
+
+mkdir -p /opt/spire/data/agent
+mkdir -p /var/run/spire
+
+if [ ! -f /opt/spire/agent.conf ]; then
+  echo "Writing default SPIRE agent.conf..."
+  cp /etc/spire/templates/agent.conf /opt/spire/agent.conf
+  chmod 644 /opt/spire/agent.conf
+fi
+
+exec /usr/local/bin/spire-agent "$@"

spire/entrypoint.server.sh

@@ -0,0 +1,13 @@
+#!/bin/sh
+
+set -e
+
+mkdir -p /opt/spire/data/server
+
+if [ ! -f /opt/spire/server.conf ]; then
+  echo "Writing default SPIRE server.conf..."
+  cp /etc/spire/templates/server.conf /opt/spire/server.conf
+  chmod 644 /opt/spire/server.conf
+fi
+
+exec /usr/local/bin/spire-server "$@"

spire/templates/agent.conf

@@ -0,0 +1,51 @@
+# ==============================================================================
+# SPIRE Agent Configuration
+# Auth-Yes Identity & Access Management Fabric
+# ==============================================================================
+# The SPIRE Agent runs as a local node daemon, attesting workloads (e.g. Docker
+# containers) and serving the Workload API UNIX domain socket.
+# ==============================================================================
+
+agent {
+  # Directory where the SPIRE agent caches SVIDs, bundles, and keys.
+  data_dir = "/opt/spire/data/agent"
+
+  # Logging verbosity: DEBUG, INFO, WARN, ERROR
+  log_level = "INFO"
+
+  # Address and port of the SPIRE Server container within the internal network.
+  server_address = "spire-server"
+  server_port = "8081"
+
+  # UNIX Domain Socket Path exposed to workloads for zero-trust identity fetching.
+  socket_path = "/var/run/spire/agent.sock"
+
+  # Must match the SPIRE Server's trust_domain.
+  trust_domain = "system.local"
+}
+
+plugins {
+  # ----------------------------------------------------------------------------
+  # NodeAttestor Plugin: Authenticates this agent with the SPIRE Server.
+  # ----------------------------------------------------------------------------
+  NodeAttestor "join_token" {
+    plugin_data {}
+  }
+
+  # ----------------------------------------------------------------------------
+  # KeyManager Plugin: Stores node-level private keys on disk.
+  # ----------------------------------------------------------------------------
+  KeyManager "disk" {
+    plugin_data {
+      directory = "/opt/spire/data/agent"
+    }
+  }
+
+  # ----------------------------------------------------------------------------
+  # WorkloadAttestor Plugin: Inspects running containers on Docker / Podman
+  # to grant SPIFFE SVIDs based on container image, labels, or names.
+  # ----------------------------------------------------------------------------
+  WorkloadAttestor "docker" {
+    plugin_data {}
+  }
+}

spire/templates/server.conf

@@ -0,0 +1,59 @@
+# ==============================================================================
+# SPIRE Server Configuration
+# Auth-Yes Identity & Access Management Fabric
+# ==============================================================================
+# This configuration defines the root SPIFFE trust authority for your cluster.
+# You can customize trust domains, certificate TTLs, and datastore plugins below.
+# ==============================================================================
+
+server {
+  # Network binding: 0.0.0.0 listens on all internal mesh interfaces.
+  bind_address = "0.0.0.0"
+  bind_port = "8081"
+
+  # Trust Domain: Identifies the root cryptographic security domain.
+  # SPIFFE IDs will be generated in the format: spiffe://<trust_domain>/<workload>
+  trust_domain = "system.local"
+
+  # Directory where SPIRE server persists runtime data, datastore, and keys.
+  data_dir = "/opt/spire/data/server"
+
+  # Logging verbosity: DEBUG, INFO, WARN, ERROR
+  log_level = "INFO"
+
+  # Certificate Authority (CA) Time-to-Live (default: 30 days)
+  ca_ttl = "720h"
+
+  # Default Workload SVID Time-to-Live (default: 1 hour for high-security rotation)
+  default_x509_svid_ttl = "1h"
+}
+
+plugins {
+  # ----------------------------------------------------------------------------
+  # DataStore Plugin: Persists SPIFFE registrations, entries, and nodes.
+  # Default: Embedded SQLite3 datastore inside /opt/spire/data/server.
+  # ----------------------------------------------------------------------------
+  DataStore "sql" {
+    plugin_data {
+      database_type = "sqlite3"
+      connection_string = "/opt/spire/data/server/datastore.sqlite3"
+    }
+  }
+
+  # ----------------------------------------------------------------------------
+  # NodeAttestor Plugin: Verifies identity of SPIRE agents joining the cluster.
+  # 'join_token' allows dynamic 1-time token enrollment for agents.
+  # ----------------------------------------------------------------------------
+  NodeAttestor "join_token" {
+    plugin_data {}
+  }
+
+  # ----------------------------------------------------------------------------
+  # KeyManager Plugin: Securely stores the server CA private keys on disk.
+  # ----------------------------------------------------------------------------
+  KeyManager "disk" {
+    plugin_data {
+      keys_path = "/opt/spire/data/server/keys.json"
+    }
+  }
+}