NixOS x Proxmox infra configuration with Terranix
This repository allows to manage LXC containers on Proxmox.
It's supposed to be used from an host where nixos-rebuild is available.
- Uses
nix-community/generatorsto build a LXC template with a base NixOS container - Uses
terranixto build the infra definition andopentofuto deploy it on Proxmox - Uses
nixos-rebuildto deploy the configuration on the container - Uses
agenixto avoid having too many secrets in clear
My main objective was to have a "light" definition for the containers and to be able to use Nix to factorize configuration.
Sadly as there is quite a bit of self/cross-references, I "had" to frequently split the configs in two definitions (which pretty much defeats my initial purpose).
Architecture
config/_globals.nix=> Global values used in a lot of placesconfig/_ids.nix=> hostname -> container ID mappingconfig/_keys.nix=> hostname -> public ssh key mapping (for secrets, built with ssh-keyscan {ip})config/_passwords.nix=> hostname -> db password...containers/{name}.nix=> main definition of the container with my custom modulemodules/containers.nix=> container module definition & evaluationmodules/tools.nix=> some utilities using global config etc. Also the main logic to convert a container ID to an IPconfig/{name}-*.nix=> additional configuration if needed, imported with the whole config available for cross-references.secrets/=> Agenix secrets
My Stack
- DNS: AdGuardHome with Unbound
- Reverse Proxy: Traefik
- Auth: Authentik - mostly manual for now, I'd like to add Terraform...
- DB: Postgres
- Monitoring: Prometheus + Loki + Grafana and Alloy agents on hosts
Setting up
- Build your
config/_globals.nixusing the example - Fill in
config/_ids.nix- probably starting with your existing and relevant hosts/IP - Adapt
modules/tools.nixto be relevant with your setup - Adapt
modules/lxc-template.nixas needed - Run
build-template(=nix build .#lxc-template -o nixos-template) to build the Proxmox LXC template. - Upload the template to Proxmox (TODO Try to automate this)
- Create a user/role etc on Proxmox (see the provider documentation)
- Create
terraform.tfvarsand adapt it with the relevant infos - Run
tofu init
Probably remove all files in containers/ that you don't need, or move them to another folder. Another option is to set my-lxc.[name].container.enable = false;.
Also it is tailored for my use and was built iteratively so I guess if you want to replicate my setup you'll have to disable things to make it work and iteratively add them back. First I'd disable logging.enable in every my-lxc module.
Adding a container
To add a container, the main workflow is:
- run
add-lxc {name} {id} - edit
containers/{name}.nix- adapt container specs
- setup NixOS config
- setup DB if needed
- enable logging, routing, auth, ...
- if needed, add additional definition in
config/{name}-{service}.nixand reference it incontainers/{name}.nix - add needed secrets in
secrets/secrets.nixand usecd secrets && agenix -e {secret-name}.ageto create the encrypted files (see agenix doc) - run
build-terraform-jsonto build the Terraform definition - run
tofu apply, REVIEW the plan, and confirm if it's OK - start the container on Proxmox (probably should add the option in terraform but I prefer to have control...)
- run
ssh-keyscan {ip}and add the public SSH key toconfig/_keys.nixfor the secrets decoding - runcd secrets && agenix -rto re-build the secrets for your new key - run
deploy-lxc {name}to deploy the NixOS setup to Proxmox - if needed, run
deploy-lxc {proxy}to enable routing to your new container