Victor's room

Overview & Root Cause Summary: The error MISCONF Redis is configured to save RDB snapshots, but is currently not able to persist on disk. Commands that may modify the data set are disabled occurs when Redis fails to complete a background snapshot (BGSAVE) to disk. By default, Redis protects data integrity by refusing all write operations (such as SET, HSET, or LPUSH) until background persistence succeeds. The primary root causes include Linux kernel memory overcommit restrictions, exhausted server disk space, or incorrect filesystem permissions on the Redis data directory.

Understanding the Root Causes

• Strict Linux Kernel Memory Overcommit (vm.overcommit_memory = 0): When Redis persists data, it calls fork() to create a child process. Under Linux’s default heuristic overcommit mode (0), the kernel rejects the fork if the server’s available RAM is smaller than the Redis process size, even though Linux uses Copy-On-Write (COW).
• Disk Space Exhaustion: The storage partition hosting the Redis database file (dump.rdb) has reached 100% capacity, preventing the child process from writing snapshot buffers.
• Data Directory Permission Mismatch: The working directory specified by the dir directive (typically /var/lib/redis) is not writable by the redis system user.
• Orphaned or Stale Background Save Process: A previous background save operation hung or was killed uncleanly, leaving an unreleased lock file.

Step 1: Emergency Quick Fix via redis-cli (Disable Stop-Writes)

Temporarily unblock write operations in production while investigating underlying disk and memory issues.

# 1. Connect to your Redis instance:
redis-cli

# If authentication is required:
# AUTH your_redis_password

# 2. Dynamically disable the write-blocking safety mechanism:
CONFIG SET stop-writes-on-bgsave-error no

# 3. Test that writes are immediately accepted again:
SET test_key "test_value"
# Output: OK

# Warning: This is an emergency stopgap to restore application traffic. 
# You must resolve the memory or disk root cause to ensure data persistence!

Step 2: Enable Linux Kernel Memory Overcommit (vm.overcommit_memory = 1)

Instruct the Linux kernel to allow memory overcommit, ensuring Redis background forks succeed without allocating duplicate physical memory.

# 1. Check current overcommit memory setting:
sysctl vm.overcommit_memory
# Output: vm.overcommit_memory = 0

# 2. Immediately enable overcommit memory (runtime):
sudo sysctl vm.overcommit_memory=1

# 3. Persist the setting across system reboots:
echo "vm.overcommit_memory = 1" | sudo tee -a /etc/sysctl.conf

# 4. Reload sysctl configuration:
sudo sysctl -p

Step 3: Resolve Disk Space Exhaustion & Working Directory Permissions

Verify disk capacity and ensure the redis user has ownership of the persistence directory.

# 1. Check available disk space across partitions:
df -h

# 2. Identify the configured Redis persistence directory:
redis-cli CONFIG GET dir
# Example output: 1) "dir" 2) "/var/lib/redis"

# 3. Verify and fix directory ownership:
sudo chown -R redis:redis /var/lib/redis
sudo chmod 755 /var/lib/redis

# 4. If disk is full, clear system journal logs or rotate old log files:
sudo journalctl --vacuum-size=500M

Verification & Testing Steps

Re-enable persistence safeguards, trigger a manual snapshot, and verify background persistence status.

# 1. Re-enable write protection in redis-cli:
CONFIG SET stop-writes-on-bgsave-error yes

# 2. Trigger an explicit background save:
BGSAVE
# Expected output: Background saving started

# 3. Check persistence status metrics:
INFO persistence

# Verify that:
# rdb_last_bgsave_status:ok
# rdb_bgsave_in_progress:0

# 4. Confirm the timestamp of the latest successful snapshot:
LASTSAVE

Summary Comparison Table