Deployment Troubleshooting

This guide helps resolve common issues when deploying or running TurboPanel in production. For security-related issues, see Security. For setup verification, see Control plane and Daemon setup.

Introduction

Production issues typically fall into TLS/Caddy misconfiguration, instance socket permissions, daemon WebSocket connectivity, Postgres socket access, or Ansible playbook failures. Self-hosted browser entrypoint is HTTPS on port 8443 (Caddy). On co-located dev hosts only, Caddy also serves a plaintext mirror at http://<host>:8880 when TURBOPANEL_DEV_HTTP_CONTROL_PLANE=1.

Port conflicts (8443 / 8880)

Problem: Caddy cannot bind :8443 or (dev) :8880.

Solutions:

1. Change CADDY_PORT / CADDY_HTTP_PORT in instance/Caddy env and update firewall rules
2. Check for existing listeners: ss -tlnp | grep -E '8443|8880'
3. Stop conflicting services or remap the published port in your reverse proxy

UI or API not loading

Problem: Browser shows errors at https://<host>:8443 or (co-located dev) http://<host>:8880.

Solutions:

1. Check instance: systemctl status turbopanel-instance
2. Check Caddy: systemctl status turbopanel-caddy
3. Test health over HTTPS: curl -k https://localhost:8443/api/health
4. Co-located dev only: test plaintext mirror: curl http://localhost:8880/api/health (requires TURBOPANEL_DEV_HTTP_CONTROL_PLANE=1; 403 when the flag is off)
5. Logs: journalctl -u turbopanel-instance -u turbopanel-caddy -f
6. Static UI: ensure ui/dist exists if TURBOPANEL_UI_MODE=static
7. Dev UI: ensure turbopanel-ui is running when TURBOPANEL_UI_MODE=dev

Daemon shows offline but service is running

Problem: UI badge is Offline (or Update fails) while systemctl status turbopaneld is active and the Cell panel shows recent activity.

Cause: The server list uses a Postgres presence projection that can drift from live WebSocket state after a failed upgrade or reconnect.

Solution: Re-run the release installer — see Refresh a stuck daemon.

Daemon not connecting

Problem: Remote server daemon fails to register.

Solutions:

1. Verify instance URL: TURBOPANEL_INSTANCE_URL=https://<host>:8443 (production) or, on co-located dev only, http://<host>:8880 when TURBOPANEL_DEV_HTTP_CONTROL_PLANE=1 (no CA required)
2. Test from node over HTTPS: curl -k https://<host>:8443/api/daemon/v1/version
3. Co-located dev only: test plaintext: curl http://<host>:8880/api/daemon/v1/version
4. Confirm platform CA (HTTPS path): compare with GET /api/daemon/v1/instance/ca
5. Firewall: allow 8443 from remote daemons to instance (8880 is dev-only and not for remote production nodes)
6. Logs: journalctl -u turbopaneld -f

Postgres connection failures (self-hosted)

Problem: Instance cannot connect to Postgres.

Solutions:

1. Check Postgres container: docker ps | grep postgres
2. Verify Unix socket under /var/run/turbopanel/postgres/
3. Confirm TURBOPANEL_DATABASE_URL in turbopanel-instance.service — a full Postgres URL, typically injected by the instance-launch role
4. Socket directory permissions: /run/turbopanel should be 2770 tp:tp

Install wizard / PAM failures (Deno)

Problem: Install bootstrap rejects host credentials.

Solutions:

1. Install pamtester on the host
2. Confirm tpctrl user sudoers allows pamtester login * authenticate
3. Use a host account in sudo / wheel / admin, or root

UI Update stuck on "update already in progress"

Problem: After clicking Update, Ansible runs but the daemon keeps the same PID for a long time. A second update attempt returns update already in progress.

Cause: The UI update path uses run.sh --no-start (does not stop the daemon during reconcile). The daemon then tries to restart itself with plain systemctl enable --now as the tp user — that fails silently (non-root cannot restart systemd units; enable --now also skips restart when already active). An in-memory lock then blocks further updates.

Check logs on the server:

sudo tail -30 /var/log/turbopanel/daemon.err.log
sudo grep -i 'systemctl\|sudo\|update' /var/log/turbopanel/daemon.log | tail -20

Immediate recovery on the server:

sudo systemctl restart turbopaneld

Or reconcile by re-running run.sh — see Refresh a stuck daemon.

Ansible / upgrade failures

Problem: Upgrade System or daemon orchestration fails.

Solutions:

1. Check daemon logs — on managed hosts Ansible runs as the tp user; on co-located dev it runs as the current dev user
2. Dirty git checkouts block upgrade; commit or stash changes in $HOME source repos (~/daemon, ~/instance, ~/ui, ~/website)

General debugging

Related documentation

• Security — TLS and authentication
• Control plane — Service layout
• Daemon setup — Node installer
• Dev console troubleshooting — Local dev issues