feat: massive refactoring...

README.md

@@ -7,46 +7,63 @@ It's supposed to be used from an host where `nixos-rebuild` is available.
   - Uses `nix-community/generators` to build a LXC template with a base NixOS container
   - Uses `terranix` to build the infra definition and `opentofu` to deploy it on Proxmox
   - Uses `nixos-rebuild` to deploy the configuration on the container
+  - Uses `agenix` to avoid having too many secrets in clear
 
 My main objective was to have a "light" definition for the containers and to be able to use Nix to factorize configuration.
 
-# Usage
+Sadly as there is quite a bit of self/cross-references, I "had" to frequently split the configs in two definitions (which pretty much defeats my initial purpose).
 
-## Prepare the infra constants
-- `cp lib/constants.nix.template lib/constants.nix`
-- adapt `lib/constants.nix` to match your needs
-- touch `lib/ips.nix`
-- remove both these files from `.gitignore` and `git add` them. 
+# Architecture
 
-## Build NixOS template
-- modify `lib/lxc-template.nix` as needed
-- run `build-template`
-- template available in `nixos-template/tarball/`
-(.tar.xz to be uploaded to Proxmox)
+- `config/_globals.nix` => Global values used in a lot of places
+- `config/_ids.nix` => hostname -> container ID mapping
+- `config/_keys.nix` => hostname -> public ssh key mapping (for secrets, built with ssh-keyscan {ip})
+- `config/_passwords.nix` => hostname -> db password...
+- `containers/{name}.nix` => main definition of the container with my custom module
+- `modules/containers.nix` => container module definition & evaluation
+- `modules/tools.nix` => some utilities using global config etc. Also the main logic to convert a container ID to an IP
+- `config/{name}-*.nix` => additional configuration if needed, imported with the whole config available for cross-references.
+- `secrets/` => Agenix secrets
 
-TODO Script the Proxmox Template upload if possible.
+# My Stack
 
-## Prepare Terraform
-- create a user/role etc on Proxmox (see [the provider documentation](https://registry.terraform.io/providers/Telmate/proxmox/latest/docs))
-- `cp terraform.tfvars.example terraform.tfvars`
-- edit `terraform.tfvars` to fill in values
-- adapt the terraform base config as needed in `infra/main.nix`
-- run `tofu init`
+- DNS: AdGuardHome with Unbound
+- Reverse Proxy: Traefik
+- Auth: Authentik - mostly manual for now, I'd like to add Terraform...
+- DB: Postgres
+- Monitoring: Prometheus + Loki + Grafana and Alloy agents on hosts
 
-## Adapt NixOS / Terraform modules building
-- edit `lib/container_build.nix` to change how a container definition is translated to TF / NixOS config (in particular check the template name)
+# Setting up
 
-## Create containers definitions
-- run `add-lxc [name] [id]`
-- edit `lxc/#NAME#.nix` as needed
-- run `build-terraform-json`
-- run `tofu plan` and review the plan
-- run `tofu apply`, hopefully without errors
-- run `deploy-lxc #NAME#`
+1. Build your `config/_globals.nix` using the example
+2. Fill in `config/_ids.nix` - probably starting with your existing and relevant hosts/IP
+3. Adapt `modules/tools.nix` to be relevant with your setup
+4. Adapt `modules/lxc-template.nix` as needed
+5. Run `build-template` (= `nix build .#lxc-template -o nixos-template`) to build the Proxmox LXC template.
+6. Upload the template to Proxmox (TODO Try to automate this)
+7. Create a user/role etc on Proxmox (see [the provider documentation](https://registry.terraform.io/providers/Telmate/proxmox/latest/docs))
+8. Create `terraform.tfvars` and adapt it with the relevant infos
+9. Run `tofu init`
 
-## Update container
-- edit `lxc/#NAME#.nix` as needed
-- if the container specs have changed, do all as above
-- otherwise you can just run `deploy-lxc #NAME#`
+Probably remove all files in `containers/` that you don't need, or move them to another folder. Another option is to set `my-lxc.[name].container.enable = false;`.
 
+Also it is tailored for my use and was built iteratively so I guess if you want to replicate my setup you'll have to disable things to make it work and iteratively add them back. First I'd disable `logging.enable` in every `my-lxc` module.
+
+# Adding a container
+
+To add a container, the main workflow is:
+- run `add-lxc {name} {id}`
+- edit `containers/{name}.nix`
+  - adapt container specs
+  - setup NixOS config
+  - setup DB if needed
+  - enable logging, routing, auth, ...
+- if needed, add additional definition in `config/{name}-{service}.nix` and reference it in `containers/{name}.nix`
+- add needed secrets in `secrets/secrets.nix` and use `cd secrets && agenix -e {secret-name}.age` to create the encrypted files (see agenix doc)
+- run `build-terraform-json` to build the Terraform definition
+- run `tofu apply`, REVIEW the plan, and confirm if it's OK
+- start the container on Proxmox (probably should add the option in terraform but I prefer to have control...)
+- run `ssh-keyscan {ip}` and add the public SSH key to `config/_keys.nix` for the secrets decoding - run `cd secrets && agenix -r` to re-build the secrets for your new key
+- run `deploy-lxc {name}` to deploy the NixOS setup to Proxmox
+- if needed, run `deploy-lxc {proxy}` to enable routing to your new container