Post-CTF Infrastructure Analysis

1. Infrastructure Overview

VMs

CTFd VM Stack (mainctfd-1)

Stock infra from CTFd opensource a bit modified(some hashing funcs n all, mentioned at the end)

Runner 1 Stack (ctf-instance)

Runner 2 Stack (instancer2)

Networking

Players
  │
  ├── ctf.themctf.com (Cloudflare Tunnel → mainVM)
  │
  └── runner1(frps HTTP — web challenges)
  Other chals for players were mostly single instanced 

WireGuard VPN 
  mainctfd-1 ←→ ctf-instance ←→ instancer2 

CTFd Plugins

There was other shit as well not important.

2. Architecture Diagram

┌────────────────────────────────────────────────────────────────────┐
│                        THEM?!CTF 2026                              │
│                                                                    │
│  Players ──► Cloudflare ──► Caddy ──► CTFd (mainctfd-1)            │
│                                         │                          │
│                                    WireGuard VPN                   │
│                                    10.8.0.0/24                     │
│                                         │                          │
│              ┌──────────────────────────┴──────────────────┐       │
│              │                                             │       │
│         ctf-instance                    instancer2         │       │
│         Runner 1 — Swarm Leader            Runner 2 — Swarm Worker │
│              │                                             │       │
│         ┌────┴────┐                               ┌────────┘       │
│         frps      Registry                    Whale instances      │
│         frpc                                  (random placement)   │
│         chall-mgr                                                  │
│                                                                    │
└────────────────────────────────────────────────────────────────────┘

3. Incidents & Issues

Another minor incident was that the FRPC was constantly gettin fucked up, so we had to kill ghost containers manually and smtimes configs were overwritten so we had to fix that everytime.

• The cause for config overwrites go back to the same old someone running something on local with VPN enabled.

4. Security Hardening

CTFd VM

Runner 1

Runner 2

ELK Stack

Elk stack was sestup on all 3 vms with proper data ingestion. Rules were in place too for alerts and was integrated with discord for overlooking. Thank God no security incident took place.

5. Performance Observations

CTFd VM Resource Usage (Peak)

other containers abd vms were pretty normal throughout

6. Tech Stack

For the final question? TECH STACK

Cloud & Infrastructure

• Multiple VM infra across 3 nodes (CTFd VM + 2 runner VMs)
• WireGuard VPN mesh
• Cloudflare Tunnel for zero-exposure public access.
• Docker Swarm multi-node cluster with dynamic workload distribution
• Private Docker registry with TLS

Container Orchestration & Security

• Docker Swarm with placement constraints
• Per-team isolated challenge containers via ctfd-whale plugin
• Container hardening
• Network isolation via iptables DOCKER-USER chain and raw table rules
• Resource limits (CPU/memory) per challenge container

Networking

• frp (Fast Reverse Proxy) for HTTP subdomain routing and TCP port forwarding
• iptables multi-layer ruleset (DOCKER-USER + raw PREROUTING)
• UFW firewall with interface-scoped rules (WireGuard-only for sensitive ports)

Platform & Backend

• CTFd 3.8.1 with custom plugins (ctfd-whale, ctfd-chall-manager)
• Rest was stock.
• Dynamic flags for instanced chals.

Security Operations

• Elastic Stack SIEM across all 3 VMs
• Discord-integrated alerting
• Incident response and Snapshot Plan

Languages & Tools

• Bash, Python, SQL (MariaDB)
• iptables, ufw, wireguard, docker, docker compose
• Kibana KQL, Elastic Agent policy management
• Git, VS Code

So yea guys thats it for this time ig. If you ask me what i found the hardest? prolly debugging in prod. this is smthg i'd like to not do again(especially with so many tickets being created and constant discord notifs.)

some of you might be wondering did i slop this too? nah. The log part was summarised by slop but i verified it manually and removed the non-important ones.