Safely Update Environment Variables Across Multiple Production Servers

The Check-Update-Verify Pattern

When you need to update configuration values like API credentials across multiple production servers, a systematic approach prevents costly mistakes. Here’s a three-step pattern that gives you full visibility and confidence:

Step 1: Check Current State

Before making changes, audit what’s currently deployed across all servers. Use SSH with grep to read environment variables:

ssh web01 "grep '^API_KEY=' /var/www/app/.env"
ssh web02 "grep '^API_KEY=' /var/www/app/.env"
ssh api01 "grep '^API_KEY=' /var/www/app/.env"

This reveals discrepancies immediately. You might discover that some servers already have the updated value, or that different servers are using different credentials entirely.

Step 2: Update with Precision

Use sed to make surgical changes without touching other configuration values:

ssh web01 "sed -i 's/^API_KEY=.*/API_KEY=\"sk_live_abc123xyz\"/' /var/www/app/.env"
ssh web02 "sed -i 's/^API_KEY=.*/API_KEY=\"sk_live_abc123xyz\"/' /var/www/app/.env"
ssh api01 "sed -i 's/^API_KEY=.*/API_KEY=\"sk_live_abc123xyz\"/' /var/www/app/.env"

The caret (^) anchor is crucial here—it ensures you only match lines that start with API_KEY=, preventing accidental modifications elsewhere in the file where that string might appear in comments or other contexts.

Step 3: Verify Success

Run the same grep command from step 1 across all servers again to confirm consistency:

ssh web01 "grep '^API_KEY=' /var/www/app/.env"
ssh web02 "grep '^API_KEY=' /var/www/app/.env"  
ssh api01 "grep '^API_KEY=' /var/www/app/.env"

All servers should now return identical output. If any server differs, you caught it before it causes problems.

When to Use This Pattern

This approach shines when:

• You don’t have configuration management tools like Ansible or Puppet set up
• You need to make a one-off change quickly without going through a full deployment pipeline
• You’re working in an environment where you’re sudoer on production servers
• The number of servers is small enough that manual SSH is practical (typically under 10-15 servers)

The pattern trades automation for visibility and control. You see exactly what’s happening at each step, which is valuable when working with sensitive configuration like API credentials or database passwords.

Pro Tips

Escape quotes properly: When the value contains quotes, escape them in the sed command: \"value-here\"

Use a consistent naming pattern: If your .env file has similar variable names like API_KEY, API_KEY_SANDBOX, and PARTNER_API_KEY, the ^ anchor prevents accidentally matching the wrong one.

Test on one server first: If you’re unsure about the sed syntax, run it on one server, verify the result, then proceed to the others.

Consider a for loop: For many servers, wrap it in a loop:

for server in web01 web02 api01; do
    echo "Updating $server..."
    ssh "$server" "sed -i 's/^API_KEY=.*/API_KEY=\"new-value\"/' /var/www/app/.env"
    ssh "$server" "grep '^API_KEY=' /var/www/app/.env"
done

This pattern isn’t a replacement for proper configuration management, but it’s a pragmatic technique for those moments when you need to make a quick, safe change across a handful of servers without the overhead of a full deployment pipeline.