70 lines
3.8 KiB
Markdown
70 lines
3.8 KiB
Markdown
# NixOS x Proxmox infra configuration with Terranix
|
|
|
|
This repository allows to manage LXC containers on Proxmox.
|
|
|
|
It's supposed to be used from an host where `nixos-rebuild` is available.
|
|
|
|
- Uses `nix-community/generators` to build a LXC template with a base NixOS container
|
|
- Uses `terranix` to build the infra definition and `opentofu` to deploy it on Proxmox
|
|
- Uses `nixos-rebuild` to deploy the configuration on the container
|
|
- Uses `agenix` to avoid having too many secrets in clear
|
|
|
|
My main objective was to have a "light" definition for the containers and to be able to use Nix to factorize configuration.
|
|
|
|
Sadly as there is quite a bit of self/cross-references, I "had" to frequently split the configs in two definitions (which pretty much defeats my initial purpose).
|
|
|
|
# Architecture
|
|
|
|
- `config/_globals.nix` => Global values used in a lot of places
|
|
- `config/_ids.nix` => hostname -> container ID mapping
|
|
- `config/_keys.nix` => hostname -> public ssh key mapping (for secrets, built with ssh-keyscan {ip})
|
|
- `config/_passwords.nix` => hostname -> db password...
|
|
- `containers/{name}.nix` => main definition of the container with my custom module
|
|
- `modules/containers.nix` => container module definition & evaluation
|
|
- `modules/tools.nix` => some utilities using global config etc. Also the main logic to convert a container ID to an IP
|
|
- `config/{name}-*.nix` => additional configuration if needed, imported with the whole config available for cross-references.
|
|
- `secrets/` => Agenix secrets
|
|
|
|
# My Stack
|
|
|
|
- DNS: AdGuardHome with Unbound
|
|
- Reverse Proxy: Traefik
|
|
- Auth: Authentik - mostly manual for now, I'd like to add Terraform...
|
|
- DB: Postgres
|
|
- Monitoring: Prometheus + Loki + Grafana and Alloy agents on hosts
|
|
|
|
# Setting up
|
|
|
|
1. Build your `config/_globals.nix` using the example
|
|
2. Fill in `config/_ids.nix` - probably starting with your existing and relevant hosts/IP
|
|
3. Adapt `modules/tools.nix` to be relevant with your setup
|
|
4. Adapt `modules/lxc-template.nix` as needed
|
|
5. Run `build-template` (= `nix build .#lxc-template -o nixos-template`) to build the Proxmox LXC template.
|
|
6. Upload the template to Proxmox (TODO Try to automate this)
|
|
7. Create a user/role etc on Proxmox (see [the provider documentation](https://registry.terraform.io/providers/Telmate/proxmox/latest/docs))
|
|
8. Create `terraform.tfvars` and adapt it with the relevant infos
|
|
9. Run `tofu init`
|
|
|
|
Probably remove all files in `containers/` that you don't need, or move them to another folder. Another option is to set `my-lxc.[name].container.enable = false;`.
|
|
|
|
Also it is tailored for my use and was built iteratively so I guess if you want to replicate my setup you'll have to disable things to make it work and iteratively add them back. First I'd disable `logging.enable` in every `my-lxc` module.
|
|
|
|
# Adding a container
|
|
|
|
To add a container, the main workflow is:
|
|
- run `add-lxc {name} {id}`
|
|
- edit `containers/{name}.nix`
|
|
- adapt container specs
|
|
- setup NixOS config
|
|
- setup DB if needed
|
|
- enable logging, routing, auth, ...
|
|
- if needed, add additional definition in `config/{name}-{service}.nix` and reference it in `containers/{name}.nix`
|
|
- add needed secrets in `secrets/secrets.nix` and use `cd secrets && agenix -e {secret-name}.age` to create the encrypted files (see agenix doc)
|
|
- run `build-terraform-json` to build the Terraform definition
|
|
- run `tofu apply`, REVIEW the plan, and confirm if it's OK
|
|
- start the container on Proxmox (probably should add the option in terraform but I prefer to have control...)
|
|
- run `ssh-keyscan {ip}` and add the public SSH key to `config/_keys.nix` for the secrets decoding - run `cd secrets && agenix -r` to re-build the secrets for your new key
|
|
- run `deploy-lxc {name}` to deploy the NixOS setup to Proxmox
|
|
- if needed, run `deploy-lxc {proxy}` to enable routing to your new container
|
|
|