Nebula Overlay Network

Nebula is a scalable overlay network that provides encrypted connectivity between all VMs regardless of their physical location.

Architecture

                    ┌─────────────┐
                    │ Lighthouse  │
                    │ 10.10.10.10 │
                    └──────┬──────┘
                           │
         ┌─────────────────┼─────────────────┐
         │                 │                 │
    ┌────▼────┐      ┌────▼────┐      ┌────▼────┐
    │ VM 1    │      │ VM 2    │      │ VM 3    │
    │ 10.10.10.11    │ 10.10.10.20    │ 10.10.10.30
    └─────────┘      └─────────┘      └─────────┘

Groups

Nebula uses groups for firewall segmentation:

Group	Purpose	Can Access
`admin`	Personal devices	Everything
`infrastructure`	Core services	Each other
`projects`	Application workloads	Infrastructure
`lighthouse`	Nebula relays	-
`games`	Game servers	Isolated

Setup

1. Generate Certificate Authority

nebula-cert ca -name "Arvandor CA" -duration 87600h

This creates:

ca.crt - Certificate (share with all hosts)
ca.key - Private key (keep secure, do not commit!)

2. Generate Host Certificates

# Infrastructure VM example
nebula-cert sign -ca-crt ca.crt -ca-key ca.key \
  -name "dns" \
  -networks "10.10.10.11/24" \
  -groups "infrastructure" \
  -out-crt configs/1001/dns/dns.crt \
  -out-key configs/1001/dns/dns.key

# Application VM example
nebula-cert sign -ca-crt ca.crt -ca-key ca.key \
  -name "app-server" \
  -networks "10.10.10.50/24" \
  -groups "projects" \
  -out-crt configs/4050/app-server/app-server.crt \
  -out-key configs/4050/app-server/app-server.key

# Lighthouse
nebula-cert sign -ca-crt ca.crt -ca-key ca.key \
  -name "lighthouse" \
  -networks "10.10.10.10/24" \
  -groups "infrastructure,lighthouse" \
  -out-crt configs/1000/lighthouse/lighthouse.crt \
  -out-key configs/1000/lighthouse/lighthouse.key

3. Directory Structure

nebula/
├── ca.crt              # Certificate authority (commit this)
├── ca.key              # CA private key (DO NOT COMMIT)
├── configs/
│   ├── 1000/lighthouse/
│   │   ├── lighthouse.crt
│   │   └── lighthouse.key
│   ├── 1001/dns/
│   │   ├── dns.crt
│   │   └── dns.key
│   └── ...
└── README.md

4. Deploy with Ansible

The nebula.yml playbook deploys certificates and configuration:

ansible-playbook -i inventory.ini playbooks/nebula.yml --limit "new-vm"

Lighthouse Configuration

The lighthouse requires manual configuration (not managed by Ansible):

# /etc/nebula/config.yml on lighthouse
pki:
  ca: /etc/nebula/ca.crt
  cert: /etc/nebula/config.crt
  key: /etc/nebula/config.key

static_host_map: {}

lighthouse:
  am_lighthouse: true
  serve_dns: false

listen:
  host: 0.0.0.0
  port: 4242

punchy:
  punch: true
  respond: true

relay:
  am_relay: true

tun:
  dev: nebula1
  drop_local_broadcast: true
  drop_multicast: true

firewall:
  conntrack:
    tcp_timeout: 12h
    udp_timeout: 3m
    default_timeout: 10m

  outbound:
    - port: any
      proto: any
      host: any

  inbound:
    - port: any
      proto: any
      group: admin
    - port: any
      proto: any
      group: infrastructure
    - port: any
      proto: icmp
      host: any

IP Allocation

VMID Range	Network Segment	Last Octet
1000-1999	Management	10-19
2000-2999	Services	20-29
3000-3999	Data	30-49
4000-4999	Workloads	50-59
5000-5999	Monitoring	90-99

Example: VMID 3000 → 10.10.10.30

4.0 KiB Raw Blame History