Building a hardened image is like building a house. You need to put locks on the doors. But in our case, we put the locks on while we were still inside, and we didn't have the keys.

If your Packer build suddenly "hangs" and then fails with a timeout exactly when your security scripts run, you've likely committed Firewall Suicide.

The Problem: The "Default Drop" Trap

CIS benchmarks require a host-based firewall, usually nftables. When a hardening script enables nftables, it often applies a "Default Drop" policy.

The moment that command runs, your SSH connection (which Packer uses to communicate with the instance) is severed. Packer sits there waiting for a response that will never come, eventually timing out and destroying the build.

The Solution: The "Sandwich" Configuration

We solved this by "sandwiching" the firewall setup. Instead of letting the hardening script use a blank or restrictive configuration, we pre-inject a sane, cloud-safe nftables.conf before the service is allowed to start.

The Sane Configuration

We create a configuration that specifically allows SSH (Port 22) and essential ICMP traffic so the instance remains reachable and healthy.

# Path: /etc/nftables.conf

flush ruleset

table inet filter {
  chain input {
    type filter hook input priority 0; policy drop;
 
    # Allow loopback
    iifname "lo" accept
 
    # Allow established/related connections (essential for Packer/AWS)
    ct state established,related accept
 
    # Allow SSH (Port 22) - Keeps the builder alive
    tcp dport 22 accept

     # Allow ICMP (Ping/Diagnostics)
     icmp type echo-request accept
  }
  chain forward {
     type filter hook forward priority 0; policy drop;
  }
  chain output {
     type filter hook output priority 0; policy accept;
  }
}

The Bonus: The SELinux Relabel

While we're talking about system stability, there is one more "silent killer" in hardened builds: SELinux label mismatches. Hardening scripts modify hundreds of files, often leaving them with incorrect security labels.

In our cleanup script, we always add this line: 1 sudo touch /.autorelabel

This tells the OS to perform a full filesystem relabel on the next boot. Without it, services like D-Bus or SSH might fail to start because they don't have permission to read their own (newly hardened) config files.

The Result

Zero Build Timeouts: Our SSH connection remains stable throughout the entire hardening process.

Production Ready: The image launches with a functional firewall already in place.

Service Stability: D-Bus and other critical services start perfectly on the first boot thanks to the SELinux relabel.

Key Takeaway: Security should never be a suicide mission. Always ensure your "locks" allow the "builder" to finish the job.

Next in the series: Stop Building Everything: How to Use Smart CI/CD for Packer Pipelines.

#AWS #CyberSecurity #DevOps #Nftables #Packer #Hardening #LinuxSecurity