[System Design 1O1] Basics of Production-Grade Configuration Management (Chapter 10)

When we define Configuration Management, it is the systematic approach to organising, storing, and accessing all the settings of a backend application. You can think of this as the DNA of your application it dictates how your code behaves across different environments without changing the code itself.

While many developers immediately think of database passwords or API keys when they hear "config," that is a limited view. Relying on that definition is like saying a car is just an engine; while the engine is critical, you are missing 90% of what makes the car function. Configuration management encompasses everything from startup behaviors and feature flags to performance metrics and business rules.

The Scope of Configuration

In a modern distributed system, configuration is vast. If you are building an e-commerce platform, your configuration might include:

• Application Settings: Port numbers, log levels (debug vs. info), and connection pool sizes.
• Database Details: Hosts, ports, usernames, and timeouts.
• External Services: API keys for payment processors (like Stripe) or email providers (like Mailchimp/Resend).
• Feature Flags: Toggles to enable new features (like a new checkout flow) for specific user segments or regions.
• Business Rules: Logic parameters, such as the maximum order amount for a user.

Without a dedicated strategy, you risk "Configuration Chaos" — hard-coded values scattered throughout the codebase, security vulnerabilities from exposed secrets, and inconsistent behaviors that make debugging a nightmare.

Storage Strategies: Where Does Config Live?

Not all configurations are created equal, and where you store them matters.

1. Environment Variables: The most common method, especially for Node.js, Python, and Go apps. In local development, we often use .env files loaded into the operating system's environment.
2. Files (YAML/JSON/TOML): YAML is widely used because, unlike JSON, it supports comments, allowing teams to document why a specific config exists.
3. Cloud Secret Managers: For production systems handling sensitive data, tools like HashiCorp Vault, AWS Parameter Store, or Google Secret Manager are essential. They handle encryption at rest and in transit.

A robust production strategy often uses a hybrid approach: loading defaults from a file, overriding them with environment variables, and fetching sensitive secrets from a cloud provider.

The Environment Hierarchy

Why do we need different configs for different environments? Because each environment has a different priority.

• Development: Priority is Productivity. We use DEBUG log levels to catch issues fast and small database connection pools because local machines don't need to handle thousands of concurrent users.
• Staging: Priority is Parity. This environment should mirror production as closely as possible to catch bugs, though we might scale down resources (like pool sizes) to save on cloud costs.
• Production: Priority is Reliability, Security, and Performance. We switch logs to INFO to avoid clutter and increase connection pool sizes to handle traffic spikes.

Best Practices for Config Management

If you take one thing away from this guide, let it be Validation.

Developers often load environment variables and assume they exist. Instead, you should validate configuration immediately upon application startup. Use libraries like Zod (for TypeScript) or Go Validator to ensure all mandatory variables are present and correctly formatted. If a config is missing, the app should crash immediately rather than behaving strictly in production.

Finally, never hardcode secrets. Always follow the principle of least privilege for access control, and rotate your secrets periodically to minimize security risks.

Backend Security: The Paranoid Mindset

Security is not a feature you add at the end; it is a mindset. The goal of backend security is not just to implement tools but to become "paranoid" about your code. You must constantly ask: Where did I make an assumption?.

Attackers don't care about your clean code architecture; they care about where you assumed user input was clean, where you assumed a request came from a valid frontend, or where you assumed a user was who they claimed to be.

1. Injection Attacks: Code vs. Data

Injection attacks happen when an application confuses user data with system code.

• SQL Injection: This occurs when user input is concatenated directly into a database query. For example, if a user enters ' OR 1=1 -- into a login field, a naive string concatenation might turn a password check into a statement that is always true, granting access to the entire database.

The Fix: Parameterized Queries (Prepared Statements). Instead of combining strings, use placeholders (e.g., $1 or ?). The database driver treats the input purely as data, never as executable code.

• Command Injection: Similar to SQLi, but targets the Operating System. If you use user input to name a file in a shell command (like resizing an image using ffmpeg) without sanitization, an attacker could append ; rm -rf / to delete your server's filesystem.

The Fix: Use programming language functions that separate commands from arguments, bypassing the shell interpreter.

2. Authentication: Verifying Identity

If you can, use an established OAuth provider (like Auth0 or Clerk). Implementing stateful authentication, social logins, and account linking from scratch is complex and error-prone.

If you must manage passwords:

• Never store plain text.
• Hashing is not enough. Attackers use "Rainbow Tables" (precomputed hash lists) to reverse lookups.
• Salting is mandatory. Add a random string (salt) to the password before hashing to ensure uniqueness.
• Use Slow Algorithms. Algorithms like SHA-256 are too fast; GPUs can guess billions per second. Use Bcrypt or Argon2id, which are intentionally slow to thwart brute-force attacks.

Sessions vs. JWTs: While JWTs (JSON Web Tokens) are popular for stateless authentication, they make revocation difficult (you can't easily "delete" a token on a user's browser). For most backend applications, stateful sessions (using HTTPOnly, Secure, SameSite cookies) are preferred because they offer better control and security.

3. Authorization: Verifying Access

Authentication asks "Who are you?", while Authorization asks "What are you allowed to do?". A common vulnerability here is Broken Object Level Authorization (BOLA).

• The Scenario: A user is logged in. They request an invoice details API: /invoices/5.
• The Flaw: The backend checks if the user is logged in (Authentication) but fails to check if Invoice #5 actually belongs to that user. An attacker can simply change the ID to 6, 7, or 8 to steal other users' data.
• The Fix: Always scope database queries to the requesting user. Instead of SELECT * FROM invoices WHERE id=5, use SELECT * FROM invoices WHERE id=5 AND user_id=current_user.

4. Defence in Depth

Security requires layers. No single defence is perfect.

1. Input Validation: Sanitise everything at the entry point.
2. Rate Limiting: Protect your login endpoints from brute force. Layer it by IP, by account, and globally.
3. Security Headers: Use CSP (Content Security Policy) to prevent Cross-Site Scripting (XSS) and configure CORS properly.
4. Audit Logs: Track who accessed sensitive data and when.

Remember, security vulnerabilities fundamentally arise when data crosses a boundary, between the browser and server, or the server and the database. Identify these boundaries, stop making assumptions, and validate everything.