patx/projectpay

Deploy ProjectPay on the VPS with private socket and hardened nginx

Commit 99c3a23 · patx · Nov 3 at 5:52AM

Changeset
99c3a239e57f33cc0b3f1ebc4757ee1247eab16a
Parents
4394f3eb8c6d173b405c7e64a660afdce637685a

View source at this commit

Comments

No comments yet.

Log in to comment

Diff

diff --git a/README.md b/README.md
index 4ea6355..f3a29e1 100644
--- a/README.md
+++ b/README.md
@@ -122,6 +122,13 @@ Then copy the `whsec_...` value from the Stripe CLI output into `STRIPE_WEBHOOK_
 6. Stripe Checkout collects the payment.
 7. Stripe sends a webhook back to ProjectPay, and the payment is recorded.
 
+## Production hosting
+
+Production now runs on the existing VPS at https://payment.afterdarklabs.io,
+behind nginx and a private Unix socket. MongoDB Atlas, invoice links, sessions,
+and the existing Stripe webhook remain unchanged. Heroku's web dyno is scaled to
+zero and retained for rollback. See [VPS deployment and rollback](deploy/README.md).
+
 ## Deployment
 
 The included `Procfile` runs the app with Uvicorn:
diff --git a/deploy/README.md b/deploy/README.md
new file mode 100644
index 0000000..9caf638
--- /dev/null
+++ b/deploy/README.md
@@ -0,0 +1,131 @@
+# ProjectPay VPS deployment
+
+Migrated November 3, 2026 from Heroku app `afterdark` to the existing VPS at
+198.251.64.57. Public URL: https://payment.afterdarklabs.io.
+
+The app is still MicroPie/ASGI. No application or payment logic changed during
+this migration. `requirements-production.txt` pins the dependency versions
+observed on the working Heroku dyno. The same MongoDB Atlas database remains in
+use; projects, archived projects, payments, checkout records, webhook events,
+number counters, and server-side sessions were not moved or reset. The existing
+Stripe webhook URL and signing secret were retained.
+
+## Runtime
+
+- Service and socket: `projectpay.service`, `projectpay.socket`.
+- Source: `/opt/apps/projectpay/current`, a symlink into
+  `/opt/apps/projectpay/releases/20261103T054534Z`.
+- Virtual environment: `/opt/apps/projectpay/.venv`.
+- Dedicated service user: `projectpay`, without an interactive login shell.
+- Secrets: `/etc/projectpay/environment`, root-owned mode 0600. The production
+  environment was transferred privately from Heroku, not copied from the local
+  development `.env`. No credentials belong in this repository.
+- One Uvicorn worker, a 50% single-core CPU cap, 384 MiB memory high watermark,
+  512 MiB hard memory cap, and 64 tasks. Observed memory after verification was
+  about 50 MB; the VPS had about 2.8 GiB available before deployment.
+- Private Unix socket: `/run/projectpay/uvicorn.sock`, owner projectpay:www-data,
+  mode 0660. No public or loopback Python TCP port is opened. SocketBindDeny
+  prevents the service from explicitly binding IPv4/IPv6 ports.
+- The app trusts proxy headers only on this private transport. nginx replaces
+  forwarded headers, and only official Cloudflare networks may supply client IPs.
+- The service has read-only system/home protection, a private temporary directory,
+  no new privileges, and kernel/control-group restrictions. Invoice data remains
+  in Atlas, so the application needs no writable local data directory.
+- Both service and socket are enabled for reboot.
+
+## nginx and TLS
+
+| Source file | VPS destination |
+| --- | --- |
+| projectpay.service | /etc/systemd/system/projectpay.service |
+| projectpay.socket | /etc/systemd/system/projectpay.socket |
+| projectpay-http.conf | /etc/nginx/conf.d/projectpay.conf |
+| projectpay-cloudflare-realip.conf | /etc/nginx/snippets/projectpay-cloudflare-realip.conf |
+| projectpay.nginx.conf | /etc/nginx/sites-available/projectpay |
+
+`/etc/nginx/sites-enabled/projectpay` links to the vhost. Shared nginx.conf,
+GitMan's vhost, and Wail's vhost were not modified. Their service PIDs remained
+unchanged through migration; nginx was gracefully reloaded.
+
+The dedicated vhost enforces TLS, HSTS, no-store responses, sanitized proxy
+headers, 128 KiB form bodies, a separate 1 MiB Stripe webhook allowance, timeouts,
+rate limits, and connection limits. Webhooks have a separate higher rate bucket
+and are never retried automatically to another upstream. Existing application
+CSP, frame protection, secure cookies, CSRF, and webhook verification remain active.
+
+Invoice share links contain bearer tokens. nginx's dedicated
+`/var/log/nginx/projectpay-access.log` deliberately omits URL paths, query strings,
+cookies, request bodies, and authorization headers. Uvicorn access logs are off.
+The system nginx logrotate wildcard covers this log.
+
+The Cloudflare DNS record for payment.afterdarklabs.io is a proxied A record
+pointing to 198.251.64.57. A host-specific Cloudflare rule uses Full (strict)
+origin TLS; other host policies were preserved.
+
+Certbot certificate name: `projectpay`. Its certificate and key are under
+`/etc/letsencrypt/live/projectpay/`. Initial issuance used a temporary DNS
+challenge before traffic moved. Renewal was then reconfigured and successfully
+tested using HTTP webroot `/var/lib/projectpay-acme`; certbot.timer is enabled
+and active. Successful renewal reloads nginx. No Cloudflare API token is stored
+on the VPS. Keep both HTTP and HTTPS ACME locations available.
+
+## Verification
+
+Before cutover:
+
+- All 20 existing application security tests passed in the new runtime.
+- MongoDB connectivity and existing project/payment records were verified.
+- HTTPS admin login, secure cookie flags, dashboard, and logout passed.
+- Heroku accepted the VPS-created Mongo-backed admin session before DNS changed.
+- An existing public invoice rendered and the existing live Stripe Product was
+  verified using a read-only API request. No price, Checkout Session, or charge
+  was created by migration tests.
+- Unsigned Stripe webhook requests were rejected with HTTP 401.
+- Counts of projects, deleted projects, payments, checkout sessions, and webhook
+  events stayed unchanged during the preview checks.
+- nginx's user can connect to the private socket; an unrelated user cannot.
+
+After cutover:
+
+- Public HTTPS admin login/logout passed through Cloudflare to the VPS.
+- A correctly signed probe deliberately lacking an event ID passed signature
+  validation and received the expected HTTP 400 before any payment/event writes.
+- Certificate renewal validation passed and automatic renewal was configured.
+- GitMan and Wail remained healthy with protected config hashes and PIDs unchanged.
+- The old Heroku web formation was scaled to zero after public verification.
+  The app/config/custom domain remain for rollback. No Heroku add-ons were present.
+
+## Operations and rollback
+
+Useful checks:
+
+```sh
+systemctl status projectpay projectpay.socket
+journalctl -u projectpay --since '15 minutes ago'
+nginx -t
+systemctl status certbot.timer
+```
+
+For a source update, stage a new release directory, install any intentionally
+changed dependency pins, switch `current`, and restart projectpay.service. Run
+application/security, database, public invoice, and webhook verification before
+considering deployment complete. Keep credentials out of source archives and logs.
+
+Migration backup on the VPS:
+`/root/projectpay-releases/before-migration-20261103T054752Z/` contains the original
+nginx archive, protected file hashes/PIDs, and `cutover.json` with the previous DNS
+and Heroku formation. `/root/projectpay-releases/preview-verification.json` records
+the read-only preview results.
+
+Rollback while keeping the same shared Atlas data:
+
+1. Run `heroku ps:scale web=1 --app afterdark` from an authenticated workstation.
+2. Verify the Heroku app is healthy before redirecting traffic.
+3. Restore the proxied CNAME for payment.afterdarklabs.io to
+   `globular-baboon-um01s6bh971mrkx0uityd61d.herokudns.com`.
+4. Verify public admin access, customer payment pages, and Stripe webhooks.
+5. Stop the VPS app only after Heroku serves the public domain successfully.
+
+Do not restore an old database snapshot or delete Atlas data during hosting
+rollback. Both hosts use the same live database. The Stripe webhook already
+points at the public hostname, so it follows the DNS change in either direction.
diff --git a/deploy/projectpay-cloudflare-realip.conf b/deploy/projectpay-cloudflare-realip.conf
new file mode 100644
index 0000000..388ded9
--- /dev/null
+++ b/deploy/projectpay-cloudflare-realip.conf
@@ -0,0 +1,27 @@
+# Install as /etc/nginx/snippets/projectpay-cloudflare-realip.conf.
+# Retrieved from https://www.cloudflare.com/ips-v4 and /ips-v6 on 2026-11-03.
+# Never trust loopback, all addresses, or a caller-supplied X-Forwarded-For here.
+set_real_ip_from 173.245.48.0/20;
+set_real_ip_from 103.21.244.0/22;
+set_real_ip_from 103.22.200.0/22;
+set_real_ip_from 103.31.4.0/22;
+set_real_ip_from 141.101.64.0/18;
+set_real_ip_from 108.162.192.0/18;
+set_real_ip_from 190.93.240.0/20;
+set_real_ip_from 188.114.96.0/20;
+set_real_ip_from 197.234.240.0/22;
+set_real_ip_from 198.41.128.0/17;
+set_real_ip_from 162.158.0.0/15;
+set_real_ip_from 104.16.0.0/13;
+set_real_ip_from 104.24.0.0/14;
+set_real_ip_from 172.64.0.0/13;
+set_real_ip_from 131.0.72.0/22;
+set_real_ip_from 2400:cb00::/32;
+set_real_ip_from 2606:4700::/32;
+set_real_ip_from 2803:f800::/32;
+set_real_ip_from 2405:b500::/32;
+set_real_ip_from 2405:8100::/32;
+set_real_ip_from 2a06:98c0::/29;
+set_real_ip_from 2c0f:f248::/32;
+real_ip_header CF-Connecting-IP;
+real_ip_recursive off;
diff --git a/deploy/projectpay-http.conf b/deploy/projectpay-http.conf
new file mode 100644
index 0000000..3156f92
--- /dev/null
+++ b/deploy/projectpay-http.conf
@@ -0,0 +1,13 @@
+# Install as /etc/nginx/conf.d/projectpay.conf (http context).
+map "$request_method:$uri" $projectpay_auth_key {
+    default "";
+    "POST:/login" $binary_remote_addr;
+}
+limit_req_zone $binary_remote_addr zone=projectpay_requests:10m rate=20r/s;
+limit_req_zone $projectpay_auth_key zone=projectpay_auth:10m rate=10r/m;
+limit_req_zone $binary_remote_addr zone=projectpay_webhooks:10m rate=50r/s;
+limit_conn_zone $binary_remote_addr zone=projectpay_clients:10m;
+limit_conn_zone $server_name zone=projectpay_total:1m;
+# Invoice URLs contain bearer tokens: omit paths, query strings, and credentials.
+log_format projectpay_private '$remote_addr [$time_local] "$request_method $server_protocol" '
+    '$status $body_bytes_sent request_time=$request_time';
diff --git a/deploy/projectpay.nginx.conf b/deploy/projectpay.nginx.conf
new file mode 100644
index 0000000..0920f10
--- /dev/null
+++ b/deploy/projectpay.nginx.conf
@@ -0,0 +1,65 @@
+# Install as /etc/nginx/sites-available/projectpay and enable with a symlink.
+server {
+    listen 80;
+    server_name payment.afterdarklabs.io;
+    server_tokens off;
+    location ^~ /.well-known/acme-challenge/ {
+        root /var/lib/projectpay-acme;
+    }
+    location / { return 301 https://payment.afterdarklabs.io$request_uri; }
+}
+server {
+    listen 443 ssl;
+    http2 on;
+    server_name payment.afterdarklabs.io;
+    server_tokens off;
+    ssl_certificate /etc/letsencrypt/live/projectpay/fullchain.pem;
+    ssl_certificate_key /etc/letsencrypt/live/projectpay/privkey.pem;
+    include /etc/letsencrypt/options-ssl-nginx.conf;
+    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;
+    include /etc/nginx/snippets/projectpay-cloudflare-realip.conf;
+    client_max_body_size 128k;
+    client_header_timeout 15s;
+    client_body_timeout 30s;
+    keepalive_timeout 20s;
+    send_timeout 60s;
+    proxy_connect_timeout 5s;
+    proxy_read_timeout 60s;
+    proxy_send_timeout 30s;
+    proxy_http_version 1.1;
+    proxy_request_buffering on;
+    proxy_next_upstream off;
+    proxy_set_header Host payment.afterdarklabs.io;
+    proxy_set_header X-Real-IP $remote_addr;
+    proxy_set_header X-Forwarded-For $remote_addr;
+    proxy_set_header X-Forwarded-Proto https;
+    proxy_set_header X-Forwarded-Host payment.afterdarklabs.io;
+    proxy_set_header X-Forwarded-Port 443;
+    proxy_set_header X-Forwarded-Ssl "";
+    proxy_set_header Forwarded "";
+    proxy_set_header CF-Connecting-IP "";
+    proxy_set_header Connection "";
+    proxy_hide_header Cache-Control;
+    add_header Cache-Control "no-store" always;
+    add_header Strict-Transport-Security "max-age=31536000" always;
+    access_log /var/log/nginx/projectpay-access.log projectpay_private;
+    gzip off;
+    limit_req zone=projectpay_requests burst=80 nodelay;
+    limit_req zone=projectpay_auth burst=10 nodelay;
+    limit_req_status 429;
+    limit_conn projectpay_clients 16;
+    limit_conn projectpay_total 32;
+    limit_conn_status 429;
+    if ($request_method !~ ^(GET|HEAD|POST)$) { return 405; }
+    location ^~ /.well-known/acme-challenge/ {
+        root /var/lib/projectpay-acme;
+    }
+    location = /webhook/stripe {
+        client_max_body_size 1m;
+        limit_req zone=projectpay_webhooks burst=100 nodelay;
+        proxy_pass http://unix:/run/projectpay/uvicorn.sock;
+    }
+    location / {
+        proxy_pass http://unix:/run/projectpay/uvicorn.sock;
+    }
+}
diff --git a/deploy/projectpay.service b/deploy/projectpay.service
new file mode 100644
index 0000000..750a875
--- /dev/null
+++ b/deploy/projectpay.service
@@ -0,0 +1,37 @@
+[Unit]
+Description=ProjectPay invoice and payment app
+Requires=projectpay.socket
+After=network-online.target projectpay.socket
+Wants=network-online.target
+
+[Service]
+Type=simple
+User=projectpay
+Group=projectpay
+WorkingDirectory=/opt/apps/projectpay/current
+EnvironmentFile=/etc/projectpay/environment
+Environment=PYTHONDONTWRITEBYTECODE=1
+Environment=PYTHONUNBUFFERED=1
+ExecStart=/opt/apps/projectpay/.venv/bin/uvicorn app:app --fd 3 --workers 1 --proxy-headers --forwarded-allow-ips=* --no-access-log --no-server-header
+Restart=on-failure
+RestartSec=5
+CPUQuota=50%
+CPUWeight=25
+MemoryHigh=384M
+MemoryMax=512M
+MemorySwapMax=0
+TasksMax=64
+UMask=0077
+NoNewPrivileges=true
+PrivateTmp=true
+ProtectSystem=strict
+ProtectHome=true
+ProtectKernelTunables=true
+ProtectKernelModules=true
+ProtectControlGroups=true
+RestrictSUIDSGID=true
+RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6
+SocketBindDeny=any
+
+[Install]
+WantedBy=multi-user.target
diff --git a/deploy/projectpay.socket b/deploy/projectpay.socket
new file mode 100644
index 0000000..de3e6e8
--- /dev/null
+++ b/deploy/projectpay.socket
@@ -0,0 +1,14 @@
+[Unit]
+Description=Private ProjectPay socket for nginx
+
+[Socket]
+ListenStream=/run/projectpay/uvicorn.sock
+SocketUser=projectpay
+SocketGroup=www-data
+SocketMode=0660
+DirectoryMode=0755
+RemoveOnStop=yes
+Service=projectpay.service
+
+[Install]
+WantedBy=sockets.target
diff --git a/deploy/requirements-production.txt b/deploy/requirements-production.txt
new file mode 100644
index 0000000..44d8851
--- /dev/null
+++ b/deploy/requirements-production.txt
@@ -0,0 +1,22 @@
+# Snapshot of the Heroku production runtime, 2026-11-03.
+# Preserve behavior during the hosting migration; upgrade separately.
+anyio==4.15.1
+certifi==2026.10.23
+charset-normalizer==3.5.2
+click==8.5.0
+dnspython==2.9.0
+h11==0.16.0
+httpcore==1.0.9
+httpx==0.28.1
+idna==3.20
+Jinja2==3.1.6
+MarkupSafe==3.0.4
+micropie[all]==0.32
+multipart==2.0.1
+orjson==3.13.0
+pymongo==4.19.0
+requests==2.34.2
+stripe==16.0.1
+typing_extensions==4.16.0
+urllib3==2.8.0
+uvicorn==0.54.0