Plan: hutfx.com dev environments on hut — short subdomains, spawnable, documented, with mail
Date: 2026-11-01 (rev 5 — owner decisions incorporated: bare-username Samba shares + pass sato2, 0.6→petyapc rename, user passwords, mail pass = username, pipeline branch resolved; supersedes rev 1–4) · Author: NEO · Status: DRAFT (awaiting owner approval)
Goal
Run isolated dev copies of the hutfx.com Laravel app on hut (192.168.0.3) at short URLs sato.hutfx.com, petya.hutfx.com, testhut.hutfx.com, with per-user SSH keys, Samba-visible homes, @hutfx.com mailboxes in a dev_hutfx group, a written access guide — and one command (dev-spawn <name>) to add any further environment.
Current context / assumptions (verified live 2026-11-01)
- Host: hut = 192.168.0.3, Ubuntu 24.04, 56 cores, 62 GiB RAM (50 GiB available). Running: nginx 1.24, php8.3-fpm (one pool), PostgreSQL 16 (prod DB
hutfx), libvirt mail VM, Samba (usersato; dev shares named bare usernames per owner decision 2026-11-01), WireGuard (10.11.11.0/24 routes LAN), BIND9 authoritative for hutfx.com (/etc/bind/db.hutfx), Hermes, fail2ban-ui :8081, tokdash :55423. SSH on port 2022, not 22. LAN host 192.168.0.6 is aliasedpetyain /etc/hosts on hut, jaba and llm (verified 2026-11-01) — renamed topetyapc(T1b) to freepetyafor the dev DNS name. - Mail (verified live 2026-11-01): iRedMail on mail VM 0.131, PostgreSQL backend — DB
vmail(uservmailadmin, passaaTxAtt88wqUA7dtu20jLejUV9rpvOhg) + DBiredadmin; nomysqlbinary, usepsql. sudo on 0.131 =P4p4r4k\`` (provided by owner, verified working). NovadduserCLI (removed in this version) — mailboxes are created via SQL +doveadm. **Existing active mailboxes: admin@, neo@, petya@, postmaster@, sato@, spas@ —sato@andpetya@confirmed → onlytesthut@must be created**; existing passwords never touched (standing rule). Mailing lists: **mlmmjadmin 3.6.3 installed and available** (/opt/mlmmjadmin-3.6.3/tools/maillist_admin.py, backendbk_iredmail_sql`) — the group will be a real mlmmj list, no Postfix hack. - App: Laravel at
/var/www/hutfx(owner www-data; git needs sudo there), DB pgsql, origingithub.com/hutfx/lab-engine(old repo kept as remotehut_thereal). Canonical git flow (owner-defined 2026-11-01): hutfx.com = whatever is onmain;active_workis the middle/integration layer; every task starts a NEW descriptivefeature/*branch from currentactive_work; PR →active_work; merged only byhutfx/petya5q; after merge the next task gets a fresh branch;active_work→mainsync is done MANUALLY by sato or petya. Verified 2026-11-01: the flow already ran — PR#1 (cleaning_existing_code→active_work,6925d31) and PR#2 (active_work→main,199ed52) are merged;origin/mainhead199ed52contains all site content. T0 is a plain checkout + delete of the now-redundant localpipelinebranch. - DNS today: zone has
sato CNAME @→ public IP → production vhost. Nopetya/testhutrecords. - nginx today: production server blocks (lines 16, 32, 354 of
sites-available/hutfx-com) includesato.hutfx.com. TLS = Let's Encrypthutfx.comonly — no subdomain coverage. - Users: only
satoexists;petya,testhutto be created. Node v22.22.3, PHP 8.3, Composer present. - Naming decision (owner): the test environment's name is
testhut, nottest— "test" is too common. One name propagates everywhere: user, URL, DB, shares, mailbox. - Decision (rev 1, unchanged): host-native namespaces, not VMs/Docker — same stack as production, ~150 MB RAM per env, one-file rollback.
Naming convention (the contract — everything derives from <name>)
<name> ∈ {sato, petya, testhut} now; any new lowercase name later.
| Resource | Pattern | Example (testhut) |
|---|---|---|
| URL | http://<name>.hutfx.com |
http://testhut.hutfx.com |
| Linux user + docroot | <name> / /home/<name>/dev/hutfx (vhost serves public/) |
testhut / /home/testhut/dev/hutfx |
| Linux password | owner-set at spawn (4th dev-spawn arg) | petya=11@Avgust1212, testhut=11@Avgust1313 (sato unchanged) |
| Samba password | fixed sato2 for all dev users |
sato2 |
| SSH key | /home/<name>/.ssh/id_ed25519 (generated per user) |
comment testhut@hut-dev |
| Samba share | <name>, path /home/<name>, valid users = self |
\\192.168.0.3\testhut |
<name>@hutfx.com (password = owner-set), member of dev_hutfx@hutfx.com |
testhut@hutfx.com / pass 11@Avgust1313 |
|
| Postgres DB / role | hutfx_<name> / dev_<name> |
hutfx_testhut / dev_testhut |
| php-fpm pool / socket | pool.d/dev-<name>.conf / /run/php/php8.3-fpm-dev-<name>.sock |
dev-testhut.sock |
| nginx vhost | sites-available/dev-<name>.conf |
dev-testhut.conf |
| DNS record | <name> IN A 192.168.0.3 |
testhut IN A 192.168.0.3 |
HTTP-only, LAN/WG-only (nginx ACL allow 192.168.0.0/24; allow 10.11.11.0/24; deny all;). HTTPS on these names hits the production default cert — always use http://.
Architecture
Parameterized scripts /home/sato/bin/dev-spawn / dev-drop render everything from templates in /home/sato/dev-env/templates/: user, SSH key, Samba share, DB, pool, clone, .env, vhost, DNS record; a companion dev-mail step provisions the mailbox + group membership on the mail VM. The three environments are created by running dev-spawn three times; the fourth is one command away. Production is touched surgically in exactly two places: removing sato.hutfx.com from production server_name lines (owner decision — it becomes a dev host) and adding A records to the zone.
Browser ── http://<name>.hutfx.com ──► hut nginx :80 (vhost dev-<name>, LAN/WG ACL)
│ sato → fpm-dev-sato.sock → user sato → /home/sato/dev/hutfx (base: active_work, feature/* per task)
│ petya → fpm-dev-petya.sock → user petya → /home/petya/dev/hutfx (base: active_work, feature/* per task)
│ testhut → fpm-dev-testhut.sock → user testhut → /home/testhut/dev/hutfx (branch set on request by owner)
└─ prod (untouched): fpm.sock → www-data → /var/www/hutfx
Users connect: SSH :2022 (own key) · Samba \\192.168.0.3\<name> · IMAP/SMTP mail.hutfx.com · browser URL
Mail (0.131 iRedMail): sato@ petya@ testhut@ + group dev_hutfx@hutfx.com (mlmmj list via maillist_admin.py)
PostgreSQL 16: hutfx (prod) | hutfx_sato | hutfx_petya | hutfx_testhut
Step-by-step tasks
All on hut as sato (sudo where shown). Back up every /etc file before editing: sudo cp -a <f> /home/sato/backups/<basename>.bak-$(date +%Y%m%d%H%M).
T0 — Switch production to main (5 min) ⚠ touches prod
Owner confirmed (2026-11-01): main is up to date — production = main per the canonical flow.
sudo -n git -C /var/www/hutfx fetch origin
sudo -n git -C /var/www/hutfx checkout main
sudo -n git -C /var/www/hutfx pull --ff-only
pipeline fate DONE (2026-11-01): content was already in main via PR#1/PR#2; local branch deleted by hermes, lab-engine copies (pipeline, cleaning_existing_code) deleted by owner. hut_thereal is archive-only — never work in it; leave its stale branches untouched (owner decision). Nothing left to do in T0 beyond the checkout.
Verify: sudo -n git -C /var/www/hutfx status -sb → ## main...origin/main, clean; curl -sI https://hutfx.com/ | head -1 → unchanged status.
T1 — Production hygiene: free the sato.hutfx.com name (5 min) ⚠ touches prod
sudo sed -i 's/ sato\.hutfx\.com//g' /etc/nginx/sites-available/hutfx-com
sudo nginx -t && sudo systemctl reload nginx
Verify: sudo nginx -t → successful; grep -c "sato.hutfx" /etc/nginx/sites-available/hutfx-com → 0; curl -sI https://hutfx.com/ | head -1 → unchanged status.
T1b — Free the name petya: rename host 192.168.0.6 → petyapc (10 min) — owner decision 2026-11-01
The LAN box 192.168.0.6 (petya's PC / TTS) is aliased petya in name resolution everywhere; the bare name must be free so petya.hutfx.com (→ 192.168.0.3) has no local collision. Verified occurrences (2026-11-01): hut /etc/hosts (192.168.0.6 petya) + ~/.ssh/config (Host petya, line 25); jaba (0.2) /etc/hosts; llm (0.130) /etc/hosts (two lines). 0.6 itself was unreachable at check time (port 2022 refused) — rename its own hostname when it is back: sudo hostnamectl set-hostname petyapc + its /etc/hosts. Windows machines (owner side): hosts file / SSH configs — owner's work.
# per host (hut, jaba, llm): backup first, then
sudo sed -i 's/\(192\.168\.0\.6[[:space:]]\+\)petya$/\1petyapc/' /etc/hosts
# hut only: ~/.ssh/config 'Host petya' block header → 'Host petyapc' (IdentityFile lines unchanged)
Verify: getent hosts petya → empty on every host checked; ping -c1 petyapc → 192.168.0.6; ssh -p 2022 petyapc works from hut. petya.hutfx.com then resolves only via BIND (T2) → 192.168.0.3.
T2 — DNS: short A records → private IP (5 min)
Edit /etc/bind/db.hutfx: bump SOA serial (current 2026102202 → $(date +%Y%m%d)01, increment nn if re-run same day), replace sato IN CNAME @ with sato IN A 192.168.0.3, append:
petya IN A 192.168.0.3
testhut IN A 192.168.0.3
sudo named-checkzone hutfx.com /etc/bind/db.hutfx && sudo rndc reload hutfx.com
Verify: dig +short sato.hutfx.com @192.168.0.3 → 192.168.0.3 (same for petya/testhut). CloudNS secondary follows the serial bump automatically.
T3 — Templates (20 min) — the DRY core
Create /home/sato/dev-env/templates/. Placeholders {{NAME}}, {{USER}} (= same), {{DBPASS}} substituted by sed.
pool.conf:
[dev-{{NAME}}]
user = {{USER}}
group = {{USER}}
listen = /run/php/php8.3-fpm-dev-{{NAME}}.sock
listen.owner = www-data
listen.group = www-data
listen.mode = 0660
pm = dynamic
pm.max_children = 6
pm.start_servers = 1
pm.min_spare_servers = 1
pm.max_spare_servers = 2
php_admin_value[error_log] = /home/{{USER}}/dev/php-fpm-error.log
php_admin_flag[log_errors] = on
vhost.conf:
# dev-{{NAME}} — generated by dev-spawn. HTTP-only, LAN/WG-only.
server {
listen 80;
server_name {{NAME}}.hutfx.com;
root /home/{{USER}}/dev/hutfx/public;
allow 192.168.0.0/24;
allow 10.11.11.0/24;
deny all;
index index.php;
charset utf-8;
client_max_body_size 20M;
location / { try_files $uri $uri/ /index.php?$query_string; }
location ~ \.php$ {
try_files $uri =404;
fastcgi_pass unix:/run/php/php8.3-fpm-dev-{{NAME}}.sock;
fastcgi_index index.php;
include fastcgi_params;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
}
location ~ /\.(?!well-known).* { deny all; }
}
env.dot: (Laravel .env; APP_KEY filled by artisan)
APP_NAME="hutfx (dev-{{NAME}})"
APP_ENV=local
APP_DEBUG=true
APP_URL=http://{{NAME}}.hutfx.com
APP_KEY=
DB_CONNECTION=pgsql
DB_HOST=127.0.0.1
DB_PORT=5432
DB_DATABASE=hutfx_{{NAME}}
DB_USERNAME=dev_{{NAME}}
DB_PASSWORD={{DBPA…WORD}}
SESSION_DRIVER=database
CACHE_STORE=database
QUEUE_CONNECTION=sync
MAIL_MAILER=log
ssh-config: (per-user, own key)
Host github.com
User git
IdentityFile ~/.ssh/id_ed25519
IdentitiesOnly yes
Host hut
HostName 192.168.0.3
Port 2022
IdentityFile ~/.ssh/id_ed25519
samba-share.conf:
[{{NAME}}]
path = /home/{{USER}}
valid users = {{USER}}
browseable = yes
writable = yes
create mask = 0660
directory mask = 0770
T4 — dev-spawn script (35 min) — the deliverable that makes N envs trivial
/home/sato/bin/dev-spawn, mode 755. Usage: sudo /home/sato/bin/dev-spawn <name> [branch] [keyfile] [userpass] (branch default active_work; userpass sets the Linux password — omit for sato). Idempotent (check-then-create per step), set -euo pipefail:
#!/bin/bash
set -euo pipefail
NAME="${1:?usage: dev-spawn <name> [branch] [keyfile] [userpass]}"; BRANCH="${2:-active_work}"
T=/home/sato/dev-env/templates; USER="$NAME"; KEY="${3:-/home/$USER/.ssh/id_ed25519}"
USERPASS="${4:-}"; SMBPASS="sato2" # owner decisions 2026-11-01: fixed Samba pass for all; user pass per env
DBPASS="$(openssl rand -hex 12)"
# 1. user + dirs (skip for existing sato)
id -u "$USER" &>/dev/null || sudo adduser --disabled-password --gecos "dev env $NAME" "$USER"
sudo chmod 711 "/home/$USER"; sudo mkdir -p "/home/$USER/dev"; sudo chown -R "$USER:$USER" "/home/$USER/dev"
[ -n "$USERPASS" ] && echo "$USER:$USERPASS" | sudo chpasswd
# 2. SSH key (per-user ed25519) + ssh config — never overwrite existing files
sudo mkdir -p "/home/$USER/.ssh"; sudo chmod 700 "/home/$USER/.ssh"
[ -f "/home/$USER/.ssh/id_ed25519" ] || { [ -f "/home/sato/dev-env/keys/$NAME"_ed25519 ] && sudo cp "/home/sato/dev-env/keys/${NAME}"_ed25519{,.pub} "/home/$USER/.ssh/" || sudo -u "$USER" ssh-keygen -t ed25519 -N "" -C "$NAME@hut-dev" -f "/home/$USER/.ssh/id_ed25519"; }
[ -f "/home/$USER/.ssh/config" ] || sudo cp "$T/ssh-config" "/home/$USER/.ssh/config"
sudo chown -R "$USER:$USER" "/home/$USER/.ssh"; sudo chmod 600 "/home/$USER/.ssh/"* 2>/dev/null || true
# 3. Samba share + smb password
if ! testparm -s 2>/dev/null | grep -qi "^\[$NAME\]"; then
{ echo ""; sed "s/{{NAME}}/$NAME/g;s/{{USER}}/$USER/g" "$T/samba-share.conf"; } | sudo tee -a /etc/samba/smb.conf >/dev/null
fi
printf '%s\n%s\n' "$SMBPASS" "$SMBPASS" | sudo smbpasswd -a "$USER" >/dev/null
sudo testparm -s >/dev/null && sudo systemctl reload smbd nmbd 2>/dev/null || sudo systemctl restart smbd
# 4. postgres
sudo -u postgres psql -tAc "select 1 from pg_roles where rolname='dev_$NAME'" | grep -q 1 || {
sudo -u postgres createuser "dev_$NAME"; sudo -u postgres createdb -O "dev_$NAME" "hutfx_$NAME"; }
sudo -u postgres psql -c "ALTER USER dev_$NAME PASSWORD '$DBPASS';" >/dev/null
echo "$DBPASS" | sudo tee "/home/$USER/dev/.dbpass" >/dev/null && sudo chown "$USER" "/home/$USER/dev/.dbpass"
# 5. php-fpm pool
[ -f "/etc/php/8.3/fpm/pool.d/dev-$NAME.conf" ] || \
{ sed -e "s/{{NAME}}/$NAME/g;s/{{USER}}/$USER/g" "$T/pool.conf" | sudo tee "/etc/php/8.3/fpm/pool.d/dev-$NAME.conf" >/dev/null; }
sudo php-fpm8.3 -t && sudo systemctl reload php8.3-fpm
for i in $(seq 1 10); do [ -S "/run/php/php8.3-fpm-dev-$NAME.sock" ] && break; sleep 1; done
[ -S "/run/php/php8.3-fpm-dev-$NAME.sock" ] || { sudo systemctl restart php8.3-fpm; sleep 2; }
# 6. clone (public key must already be on GitHub — see T5; skip if exists)
[ -d "/home/$USER/dev/hutfx/.git" ] || sudo -u "$USER" GIT_SSH_COMMAND="ssh -i $KEY -o IdentitiesOnly=yes -o StrictHostKeyChecking=accept-new" git clone -b "$BRANCH" git@github.com:hutfx/lab-engine.git "/home/$USER/dev/hutfx"
# 7. .env + build
cd "/home/$USER/dev/hutfx"
[ -f .env ] || { sed -e "s/{{NAME}}/$NAME/g;s/{{USER}}/$USER/g;s/{{DBPASS}}/$DBPASS/g" "$T/env.dot" | sudo -u "$USER" tee .env >/dev/null
sudo -u "$USER" composer install --no-interaction
sudo -u "$USER" php artisan key:generate
sudo -u "$USER" php artisan migrate --force
sudo -u "$USER" php artisan storage:link
sudo -u "$USER" npm install && sudo -u "$USER" npm run build; }
# 8. nginx vhost
[ -f "/etc/nginx/sites-available/dev-$NAME.conf" ] || \
{ sed -e "s/{{NAME}}/$NAME/g;s/{{USER}}/$USER/g" "$T/vhost.conf" | sudo tee "/etc/nginx/sites-available/dev-$NAME.conf" >/dev/null
sudo ln -s "/etc/nginx/sites-available/dev-$NAME.conf" "/etc/nginx/sites-enabled/dev-$NAME.conf"; }
sudo nginx -t && sudo systemctl reload nginx
# 9. DNS record (only if missing)
if ! dig +short "$NAME.hutfx.com" @127.0.0.1 | grep -q '^192\.168\.0\.3$'; then
CUR=$(sudo awk '/; Serial/ {print $1; exit}' /etc/bind/db.hutfx)
NEW=$(date +%Y%m%d)01; [ "$NEW" = "$CUR" ] && NEW=$(date +%Y%m%d)02
sudo sed -i -E "/; Serial/ s/[0-9]{10}/$NEW/" /etc/bind/db.hutfx
echo "$NAME IN A 192.168.0.3" | sudo tee -a /etc/bind/db.hutfx >/dev/null
sudo named-checkzone hutfx.com /etc/bind/db.hutfx && sudo rndc reload hutfx.com
fi
echo "OK: http://$NAME.hutfx.com (branch $BRANCH, db hutfx_$NAME)"
echo "SMB: \\\\192.168.0.3\\$NAME samba pass: sato2"
echo "GitHub key (add with write access):"; sudo cat "/home/$USER/.ssh/id_ed25519.pub"
T5 — Register the generated key on GitHub (DECIDED 2026-11-01: deploy key under the hutfx account, write access)
Owner decision: the testhut key is registered by the owner as a deploy key with "Allow write access" on hutfx/lab-engine (Settings → Deploy keys → Add deploy key, title dev-testhut). petya's own key remains deferred — petya's env clones with /home/sato/.ssh/id_hermes_tunnel (3rd dev-spawn argument) until registered later.
Keypair already generated 2026-11-01: /home/sato/dev-env/keys/testhut_ed25519 (private, 600) + .pub (printed to owner). dev-spawn step 2 moves these to /home/testhut/.ssh/id_ed25519{,.pub} instead of generating new ones (never overwrites existing files).
Verify: after the owner adds the key: sudo -u testhut ssh -i ~testhut/.ssh/id_ed25519 -o IdentitiesOnly=yes -T git@github.com → Hi hutfx/lab-engine! You've successfully authenticated (deploy-key greeting names the repo).
T6 — dev-drop script (15 min)
sudo /home/sato/bin/dev-drop <name> [--purge]: reverse order — vhost+symlink (nginx -t, reload), pool file (reload fpm), DNS line + serial bump + rndc, Samba block (delete [$NAME] section + reload smbd), maillist_admin.py remove_subscribers from dev_hutfx, (optional --purge: dropdb/dropuser, deluser --remove-home, SQL-delete mailbox+forwardings rows + remove maildir on 0.131). Never touches prod files.
Verify: after dev-spawn tmp1 then dev-drop tmp1 --purge: dig +short tmp1.hutfx.com @127.0.0.1 → empty; ls /run/php/php8.3-fpm-dev-tmp1.sock → No such file; testparm -s | grep -i '\[tmp1\]' → empty; id tmp1 → no such user.
T7 — Spawn the three environments (15 min)
sudo /home/sato/bin/dev-spawn sato active_work "" # sato password untouched
sudo /home/sato/bin/dev-spawn petya active_work /home/sato/.ssh/id_hermes_tunnel 11@Avgust1212 # own key later, per owner
sudo /home/sato/bin/dev-spawn testhut <branch owner specifies at spawn time> "" 11@Avgust1313
sato/petya base on active_work — per the canonical flow each user creates per-task feature/* branches from it; dev-spawn only sets the initial checkout. testhut env: no fixed branch and no auto-reset — on request the owner specifies the branch (re-running dev-spawn testhut <branch> is idempotent: it fetches and checks out the new branch). sato's env reuses the existing user — dev-spawn skips user creation and never overwrites existing ssh-config/key files (sato's id_hermes_tunnel workflows unaffected).
Verify per env: curl -s -o /dev/null -w '%{http_code}' http://<name>.hutfx.com/ → 302 (→ /login); curl -s http://<name>.hutfx.com/login | grep -c '<form' → ≥1.
T8 — testhut-env on-demand reset (5 min)
No cron (owner correction 2026-11-01): the testhut env tracks whatever branch the owner specifies, on request. Helper /home/testhut/dev/update.sh <branch> (owner testhut, mode 755):
#!/bin/bash
set -euo pipefail
BR="${1:?usage: update.sh <branch>}"
cd /home/testhut/dev/hutfx
git fetch origin && git checkout "$BR" && git reset --hard "origin/$BR"
composer install --no-interaction --no-progress
php artisan migrate --force
php artisan test 2>&1 | tail -20
Run via sudo -u testhut /home/testhut/dev/update.sh <branch> whenever requested.
Verify: one manual run against a chosen branch → ends with Tests: N passed (or listed failures).
T9 — Mailbox testhut@ + dev_hutfx group (20 min, on mail VM 0.131)
Verified preconditions (2026-11-01): sato@ and petya@ exist and are active → untouched; only testhut@ created. sudo on 0.131 works (P4p4r4k\``). All commands overssh -i ~/.ssh/id_hermes_tunnel sato@192.168.0.131`.
- Create
testhut@hutfx.comvia SQL (exact schema verified; ASCII-only password — Dovecot rejects non-ASCII, known pitfall):
MP="11@Avgust1313" # owner decision 2026-11-01: mailbox password = same as user (sato@/petya@ keep theirs)
PW=$(doveadm pw -s SSHA512 -p "$MP") # {SSHA512} = stored scheme
TS=$(date +%Y.%m.%d.%H.%M.%S); MD="hutfx.com/t/e/s/testhut-$TS/" # iRedAdmin hash-shard pattern
export PGPASSWORD=aaTxAtt88wqUA7dtu20jLejUV9rpvOhg
psql -h 127.0.0.1 -U vmailadmin -d vmail <<SQL
INSERT INTO mailbox (username,password,name,maildir,domain)
VALUES ('testhut@hutfx.com','$PW','testhut','$MD','hutfx.com');
INSERT INTO forwardings (address,forwarding,domain,dest_domain,is_alias,active)
VALUES ('testhut@hutfx.com','testhut@hutfx.com','hutfx.com','hutfx.com',1,1);
SQL
echo 'P4p4r4k`' | sudo -S -p "" mkdir -p "/var/vmail/$MD"{new,cur,tmp}
echo 'P4p4r4k`' | sudo -S -p "" chown -R vmail:vmail "/var/vmail/hutfx.com/t/e/s/"
(Quota left at 0 = unlimited, matching every existing account; Postfix/Dovecot read the SQL maps live — no reload needed.) Store $MP on hut as /home/testhut/dev/.mailpass.
Verify: echo 'P4p4r4k' | sudo -S -p "" doveadm auth test testhut@hutfx.com "$MP"→pass; one test message to testhut@ lands in INBOX (IMAPuid search).
2. **Groupdev_hutfx@hutfx.com** via mlmmjadmin CLI (installed, backendbk_iredmail_sql→ writesmaillists` table + mlmmj spool + pgsql alias map):
ML=/opt/mlmmjadmin-3.6.3/tools/maillist_admin.py
echo 'P4p4r4k`' | sudo -S -p "" python3 $ML create dev_hutfx@hutfx.com name="dev_hutfx" access_policy=public
echo 'P4p4r4k`' | sudo -S -p "" python3 $ML add_subscribers dev_hutfx@hutfx.com sato@hutfx.com petya@hutfx.com testhut@hutfx.com
(Confirm exact argument syntax from the tool's usage banner before running; it lists create/add_subscribers/subscribers/delete.)
Verify: python3 $ML subscribers dev_hutfx@hutfx.com → 3 members; end-to-end: one message to dev_hutfx@hutfx.com → appears in ALL three INBOXes (IMAP check per the mailstack skill).
T10 — Access guide: doc/dev-environments.md (20 min)
Create in the repo working copy /var/www/hutfx/doc/dev-environments.md (doc dir is MD-only by spec), commit to cleaning_existing_code with the established sudo+author pattern, push to lab-engine. Exact sections:
- What this is — one paragraph: N isolated dev copies of hutfx.com on hut; each person/agent gets one via
sudo dev-spawn <name>. - Your environment table — per user: name, URL, email + group, SSH (
ssh -p 2022 <name>@192.168.0.3or VS Code Remote: HostName 192.168.0.3, Port 2022, User<name>, IdentityFile~/.ssh/id_ed25519), Samba\\192.168.0.3\<name>, mail client settings (IMAPmail.hutfx.com:993, SMTP:465, user = full email), DB name/role, branch policy. - First connection steps — hand out
.dbpass(SMB) and.mailpass(mail) values, test SSH, open the URL, set up mail. - Daily workflow — edit via VS Code Remote or SSH; browser refresh; per task:
git checkout -b feature/<descriptive-name> origin/active_work→ work → push → PR toactive_work(merged only by hutfx/petya5q; next task = fresh branch); testhut env is reset on request to whatever branch the owner names (sudo -u testhut /home/testhut/dev/update.sh <branch>) — never edit/home/testhut/dev/hutfxby hand;dev_hutfx@hutfx.comreaches all three devs. - Spawning a new env —
sudo dev-spawn <name> [branch]+ register printed key on GitHub + add mailbox/group line +dev-dropto remove. - Troubleshooting — 403 = not on LAN/WG; 502 = pool down (
sudo journalctl -u php8.3-fpm); usehttp://nothttps://; mail auth ASCII-only. Verify:git -C /var/www/hutfx log -1 --statshows the doc pushed to lab-engine; doc reachable via the docs vhost.
T11 — Wrap-up (5 min)
Record layout + dev-spawn/dev-drop/dev-mail usage in the hutfx-lab-engine-workflow and hutfx-mailstack skills. Final prod check: curl -sI https://hutfx.com/ | head -1 unchanged; smbclient -L 192.168.0.3 -U sato lists only the sato share (valid-users rule works).
Tests / validation (end-to-end)
- LAN browser →
http://sato.hutfx.com/login→ dev login page showinghutfx (dev-sato)(APP_NAME proves isolation). - Cross-env data: create a user on
testhut.hutfx.com; absent onsato.hutfx.comand prodhutfx.com. - ACL: from non-LAN/non-WG host →
http://sato.hutfx.com/→403 Forbidden. - Samba: map
\\192.168.0.3\petyawith petya's credentials → read/write;satocannot seepetya. Windows stale share list →Restart-Service Browser(known pitfall). - Keys:
sudo -u petya ssh -T git@github.com→ authenticated;git -C /home/petya/dev/hutfx push --dry-run→ no auth error. - Mail:
dev_hutfx@hutfx.commessage lands in all three INBOXes; each new mailbox can send and receive. - Prod intact:
https://hutfx.com/unchanged;sudo -u postgres psql -d hutfx -c 'select count(*) from users;'unchanged. - Spawnability:
dev-spawn tmp1→http://tmp1.hutfx.comworks;dev-drop tmp1 --purge→ zero residue (T6 verify).
Risks, tradeoffs, open questions
sato.hutfx.comrepurposed from public CNAME (→ prod) to private-IP dev host. External visitors lose it — by owner request.petya/testhutare new names, no legacy.- Private-IP records work from LAN and WG only (both route 192.168.0.0/24). Intentional.
- No TLS on dev names (LE cert =
hutfx.comonly): alwayshttp://. TLS later = certbot expand — deferred (YAGNI). - T1/T2 touch production nginx + live zone — backed up, validated before reload; revert = restore one file.
- Mail group via mlmmj (mlmmjadmin CLI, verified installed): a real mailing list — moderation/subscriber management available later if ever needed.
sato@andpetya@confirmed existing (active, live check 2026-11-01) — onlytesthut@is created; existing passwords never touched.- Passwords (owner-set 2026-11-01): Samba
sato2for all dev users; Linux users petya11@Avgust1212, testhut11@Avgust1313, sato unchanged; mailboxtesthut@password =11@Avgust1313(same as the user's, owner-set 2026-11-01). - Shared daemons: each env capped at 6 FPM workers; a runaway dev cannot starve prod. Stack divergence would be the trigger to revisit Docker — not before.
- Open questions: none remain. All resolved 2026-11-01 — mail password = user password (testhut
11@Avgust1313);pipelinebranch → delete (content verified insideorigin/mainvia PR#1/PR#2); testhut deploy key registered + read/write verified; petya key deferred; testhut env branch on request; production = main; Samba shares = bare usernames, SMB passsato2; host 0.6 →petyapc.
Rollback
dev-drop <name> per env; restore hutfx-com + db.hutfx + smb.conf from /home/sato/backups/; on 0.131: maillist_admin.py delete dev_hutfx@hutfx.com + SQL-delete the testhut mailbox/forwardings rows + remove its maildir; rndc reload + systemctl reload nginx php8.3-fpm smbd. Production app files never modified beyond T1's server_name lines.
Implementation notes
2026-11-01 17:20 — ALL THREE ENVS LIVE (T4/T5/T6/T9 DONE)
- sato, petya, testhut spawned via dev-spawn; all HTTP 200 with distinct titles, DBs migrated, Samba shares + DNS verified.
- petya/sato cloned with copy of id_hermes_tunnel (Q2: own petya key still deferred).
- dev-spawn patched: DNS record guard (no duplicate if name exists) + new records use CNAME @.
- prod vhost hutfx-com: sato.hutfx.com removed from server_name (sites-available AND sites-enabled — note: sites-enabled/hutfx-com is a plain copy, not a symlink). Backups in /home/sato/backups/.
- prod 401 = pre-existing auth_basic (9 occurrences in backup), not caused by these changes.
- Mail: testhut@ mailbox + dev_hutfx mlmmj list live; ACCESS.md emails sent to all (mail.log verified expansion).
- Remaining: dev-drop implementation; petyapc rename (needs 0.6 on); petya own GitHub key later. (proven live with testhut, 2026-11-01)
- Engine exists:
/home/sato/bin/dev-spawn,/home/sato/bin/update.sh, templates in/home/sato/dev-env/templates/, ssh snippets in/home/sato/dev-env/ssh-configs/. - nginx workers run as
sato(user sato;in nginx.conf) — fpm pool sockets must uselisten.owner/group = sato, not www-data. - vhost template needs
index index.php;(403 otherwise). - composer 2.11 installed system-wide at
/usr/local/bin/composer; node v22.22.3 copied to/opt/node-v22+ symlinks in/usr/local/bin(was sato-private before; testhut/others need it). - DNS record:
testhut IN CNAME @(matches sato pattern, works from outside LAN too). - Zone file is root:bind 640 — all edits via sudo; backups auto-written to /home/sato/backups/.
- testhut live: http://testhut.hutfx.com = 200, branch active_work, db hutfx_testhut, samba [testhut], user pass 11@Avgust1313, samba pass sato2.
- dev-drop not implemented yet (T4b); manual teardown if needed.
- ACCESS.md: dev-spawn step 14 writes
/home/<user>/ACCESS.md(what/where/how + branch rules) fromtemplates/readme.dot— first file on each Samba share; owner of the per-user access guide (T6) is now the spawn engine itself.