Practical Self-Hosting: A Lean DevOps Homelab Playbook

Practical Self-Hosting: A Lean DevOps Homelab Playbook

If you’re reading this, you probably know the drill: a humming server on your desk, a dozen containers, and a nagging itch to automate everything you touch. The dream is autonomy and speed—get code to production without relying on a cloud vendor to babysit your uptime. The reality is a messy balance of hardware, security, and repeatable processes. This article is my pragmatic blueprint: what I actually run in my homelab, why, and how you can copy-paste or adapt it for a weekend sprint.

Hook: why a lean, self-hosted stack beats tinkering with new “cool” tools

A few years back I chased the latest open-source darling for every role: Git hosting, CI, dashboards, logs, backups, and more. I ended up with a spaghetti of services, half-broken pipelines, and maintenance bills that grew with every feature flag. The turnaround from a commit to a deploy became a black box. I learned that the value isn’t chasing the hottest project; it’s having a small, reliable stack you can rebuild from scratch in a few hours, not days. The goal is a self-hosted, opinionated stack that you can manage with scripts, not a flight of fancy.

Core principles I actually follow

  • Keep it small and predictable. If it runs in a container or an LXC, you can reproduce it on another host.
  • Automate everything that’s repeatable. If you touch something once, automate the rest.
  • Do not ship one monolith. Separate concerns: git hosting, CI, registry, dashboards, backups.
  • Focus on backups and disaster recovery first. Without testable restores, you’re deluding yourself.
  • Use a lightweight, maintainable stack. If a project becomes too brittle or heavy, replace it.

Hardware and virtualization reality

You don’t need a data-center-grade rack to start. A single modest server will cover 80% of the use cases, and you can scale later.

  • Hardware baseline (good enough for most home setups):
  • 16–32 GB RAM
  • 2–4 TB NVMe for scratch/cache and containers, plus a HDD pool for backups and data
  • A multi-core CPU (ideally modern AMD/Intel)
  • Network: gigabit with a reliable router; a small switch if you expand
  • Virtualization path:
  • Proxmox VE or a simple KVM setup to host Linux VMs and LXC containers
  • Use LXC containers for lightweight services (Gitea, Portainer, Prometheus node_exporter)
  • Use a dedicated VM or container for heavier workloads (Jenkins-like CI if you must, or still keep it light with Drone)
  • Storage strategy:
  • ZFS or Btrfs for pools to simplify snapshots and rollbacks
  • Separate pool for backups (off-host or on a spare disk)
  • Networking notes:
  • A stable internal DNS, split DNS for internal vs external
  • A TLS endpoint via a reverse proxy (Nginx/Traefik) with Let’s Encrypt
  • A basic firewall (UFW or nftables) with a strict ingress rule set

Stack overview: what I actually run (and why)

  • Git hosting: Gitea
  • Why: Lightweight, easy to run in a container, simple user management, web UI, webhook support for CI
  • CI/CD: Drone (self-hosted)
  • Why: Modern, container-native pipelines, minimal footprint, good integration with Gitea
  • Reverse proxy and TLS: Traefik (or Nginx if you prefer)
  • Why: Dynamic config, Let's Encrypt automation, easy routing for multiple domains
  • Container management: Portainer (optional)
  • Why: Quick UI for managing containers, handy for day-to-day ops
  • Monitoring: Prometheus + Grafana
  • Prometheus node_exporter on hosts; Grafana dashboards; simple alerting
  • Logging: Loki (or self-hosted ELK-lite)
  • Why: Centralized logs with minimal overhead; optional if you’re starting small
  • Backup, artifact storage, and secrets:
  • Restic for backups; local snapshots plus offsite (Wasabi/S3)
  • Vault or simple encrypted secrets store (optional; start with environment vars and a minimal vault later)

A concrete, weekender plan (step by step)

Weekender goal: have a working git hosting, a CI pipeline, a reverse proxy with TLS, and a robust backup plan.

1) Prepare the host

  • Install Proxmox VE on your machine if you can; otherwise, install Ubuntu LTS on a bare metal server.
  • Create two pools: one for VMs/LXC containers, one for backups.
  • Set up an internal DNS entry (hostnames.local or your private domain) to resolve internal services.
  • Enable SSH key-based access and disable password login.

2) Build the base stack with containers

  • Create LXC containers for:
  • Gitea (git hosting)
  • Drone (CI)
  • Portainer (optional UI)
  • Prometheus node_exporter
  • Loki (logging) and Grafana
  • Create a VM for the reverse proxy if you prefer a dedicated surface for TLS termination.

3) TLS, domain, and routing

  • Point your domain’s A/AAAA record to your public IP.
  • In Traefik (or Nginx), configure TLS via Let’s Encrypt.
  • Routes:
  • git.example.local -> Gitea
  • ci.example.local -> Drone
  • grafana.example.local -> Grafana
  • Loki will be accessible as log.example.local (or integrated into Grafana)

4) Bootstrapping automation

  • Write a simple Ansible playbook to install and configure:
  • Docker/Podman, Gitea, Drone, Portainer
  • Traefik, Prometheus, Grafana, Loki
  • Restic backup job
  • Keep inventory small: one host, then scale to others if you add nodes.
  • Use secrets management (Ansible Vault or environment variables) to avoid hardcoding credentials.

5) Data layout and volumes

  • Mount persistent volumes for each service, map to container storage. For example:
  • Gitea: /var/lib/gitea
  • Drone: /data
  • Prometheus: /prometheus
  • Grafana: /var/lib/grafana
  • Ensure backups can snapshot these volumes; configure a separate backup job for each service.

6) Build a minimal CI pipeline

  • Gitea triggers a Drone pipeline on push:
  • Build step: containerized build (use a lightweight image)
  • Test step: unit tests
  • Publish step: store artifacts in a local registry (or S3-compatible storage)
  • Keep pipelines small and deterministic; no heavy payloads in CI run.

7) Logs, metrics, and dashboards

  • Enable Prometheus node_exporter on each host
  • Collect metrics from containers with cAdvisor or integrated exporters
  • Set up Grafana dashboards for:
  • CPU/memory/disk usage
  • Container health
  • Build/test metrics from Drone
  • Start Loki with a few parsable labels to keep costs reasonable

8) Backups and disaster recovery

  • Restic backup plan:
  • Daily incremental backups to local storage
  • Weekly full snapshot
  • Offsite replication to Wasabi/S3
  • Test restores quarterly. The test is the only way to know you’re not lying to yourself.
  • Verify that Gitea data, Drone pipelines, Prometheus data, and Grafana configurations have restorable backups.

9) Security hygiene and day-to-day ops

  • SSH hardening: key-based auth, disable root login
  • Firewall: restrict inbound ports to essential services
  • Application security:
  • Enable 2FA for Gitea users
  • Keep all containers updated; pin images to a rolling tag and test before update
  • Use read-only containers for critical services where possible
  • Secrets hygiene:
  • Do not commit secrets to Git
  • Use a secret store or environment-based secrets with strict access controls

Practical examples you can adapt today

  • Gitea docker-compose snippet (simplified)
  • This is a