Victor's room

Overview & Root Cause Summary: The HTTP status error 504 Gateway Time-out (upstream timed out (110: Connection timed out) while reading response header from upstream) occurs when Nginx acts as a reverse proxy or edge gateway, successfully establishes a TCP connection to a backend application server (such as PHP-FPM, Python Gunicorn/Uvicorn, Node.js, or a Docker container), but the upstream server fails to send a response within the configured proxy_read_timeout or fastcgi_read_timeout interval (default: 60 seconds).

Understanding the Root Causes

• Restrictive Nginx Read Timeout Limits: The default timeout of 60 seconds is too short for long-running operations like CSV/PDF generation, complex database migrations, batch image resizing, or AI model inference.
• Slow Database Queries Stalling the Upstream Worker: An unindexed or lock-blocked database query halts the application process, leaving the backend unable to stream HTTP headers back to Nginx.
• Upstream Worker Pool Exhaustion: In PHP-FPM or Gunicorn, all worker processes (pm.max_children or worker threads) are tied up serving existing slow requests, leaving incoming requests queued until the gateway timeout triggers.
• Unresponsive External API / Third-Party Dependencies: The backend application is waiting synchronously on an external HTTP service, payment gateway, or remote socket that is hung or dropping packets.

Step 1: Quick Fix (Increase Nginx Proxy and FastCGI Read Timeouts)

Extend the duration Nginx will wait for the upstream application to return response headers.

# 1. Open your virtual host configuration file:
# Ubuntu/Debian: /etc/nginx/sites-available/default
# CentOS/RHEL:   /etc/nginx/conf.d/default.conf
sudo nano /etc/nginx/sites-available/default

# --- For Reverse Proxy Backends (Node.js, Python Gunicorn, Docker) ---
# Add timeout directives inside the relevant 'location' or 'server' block:
location / {
    proxy_pass http://127.0.0.1:3000;
    proxy_connect_timeout 300s;
    proxy_send_timeout    300s;
    proxy_read_timeout    300s;
    send_timeout          300s;
}

# --- For PHP-FPM FastCGI Backends (WordPress, Laravel) ---
location ~ \.php$ {
    include snippets/fastcgi-php.conf;
    fastcgi_pass unix:/var/run/php/php8.2-fpm.sock;
    fastcgi_connect_timeout 300s;
    fastcgi_send_timeout    300s;
    fastcgi_read_timeout    300s;
}

# 2. Test syntax and reload Nginx:
sudo nginx -t
sudo systemctl reload nginx

Step 2: Align Upstream Backend Timeouts (PHP-FPM, Gunicorn, Node.js)

Ensure that the backend application server does not terminate before Nginx finishes waiting.

# --- For PHP-FPM (php.ini & pool config) ---
# 1. Edit php.ini (e.g. /etc/php/8.2/fpm/php.ini):
max_execution_time = 300

# 2. Edit the PHP-FPM www pool config (/etc/php/8.2/fpm/pool.d/www.conf):
request_terminate_timeout = 300s

# 3. Restart PHP-FPM:
sudo systemctl restart php8.2-fpm

# --- For Python Gunicorn ---
# Pass the timeout flag to gunicorn (default is 30s):
# gunicorn --timeout 300 --workers 4 app:app

# --- For Node.js (Express / HTTP) ---
# server.setTimeout(300000); // 5 minutes in milliseconds

Step 3: Scale Upstream Worker Capacity to Prevent Worker Starvation

If timeouts occur during traffic spikes, scale up worker concurrency to prevent queuing.

# Check active PHP-FPM worker status:
# In /etc/php/8.2/fpm/pool.d/www.conf:
pm = dynamic
pm.max_children = 50        # Increase based on server RAM (e.g. 30MB-50MB per child)
pm.start_servers = 10
pm.min_spare_servers = 5
pm.max_spare_servers = 20
pm.max_requests = 500       # Recycle workers to prevent memory leaks

# Restart PHP-FPM to apply:
sudo systemctl restart php8.2-fpm

Verification & Testing Steps

Confirm that long-running requests finish cleanly without triggering 504 status codes.

# 1. Test request duration and verify HTTP status code via curl:
curl -o /dev/null -s -w "HTTP Status: %{http_code} | Total Time: %{time_total}s\n" http://localhost/long-task

# Expected output:
# HTTP Status: 200 | Total Time: 65.340s

# 2. Monitor real-time Nginx error logs to confirm no upstream timeouts occur:
sudo tail -f /var/log/nginx/error.log | grep "504"

Summary Comparison Table

Layer	Configuration Directive	Default Value	Recommended Value
Nginx Proxy	proxy_read_timeout	60s	300s (5 minutes)
Nginx FastCGI	fastcgi_read_timeout	60s	300s (5 minutes)
PHP Engine	max_execution_time	30s	300
PHP-FPM Pool	request_terminate_timeout	0 (or 60s)	300s
Python Gunicorn	--timeout	30s	300