It started at 1 a.m. with a broken script

I was debugging a Python automation tool I had built to process CSV reports from multiple vendors. On paper, it was a simple job: fetch files, clean the data, merge them, and send a summary email.

But buried in the 600+ lines of code was the "golden rule" I had followed religiously for years:

"Don't Repeat Yourself (DRY).

I had abstracted almost everything into helper functions, generic utilities, and "reusable" logic. It felt smart… until it wasn't.

Every time I had to tweak one vendor's data pipeline, I ended up breaking another. My automation was elegant in theory, but a nightmare to maintain in reality.

That night, I broke the rule. I started repeating code where it made sense. The result? My scripts became cleaner, easier to debug, and faster to modify. Here's exactly how I did it and why this "rule-breaking" made my automation better.

Step 1: I Stopped Forcing One Function to Rule Them All

My old approach had a single process_file() function that took a bunch of parameters to handle every possible CSV format.

The function was over 50 lines long, with branching logic for each vendor. It looked something like this:

def process_file(file_path, vendor_type):
    if vendor_type == "vendor_a":
        # cleaning logic for vendor A
    elif vendor_type == "vendor_b":
        # cleaning logic for vendor B
    elif vendor_type == "vendor_c":
        # cleaning logic for vendor C
    # more branches...

Every time a new vendor was added, I had to cram in another elif block. It was DRY but bloated.

Instead, I wrote separate functions for each vendor:

def process_vendor_a(file_path):
    # specific logic for vendor A

def process_vendor_b(file_path):
    # specific logic for vendor B

Yes, it meant some "repeated" code for loading files and cleaning columns, but each function was now self-contained, readable, and safer to edit without affecting others.

Step 2: I Made Automation Pipelines Modular — But Not Over-Abstracted

Before, my automation pipeline was a giant all-purpose script. After breaking DRY, I started splitting scripts into logical, vendor-specific modules.

Example directory structure:

automation/
    vendors/
        vendor_a.py
        vendor_b.py
    utils/
        file_ops.py
    main.py

In vendor_a.py, I had:

from utils.file_ops import load_csv

def run_pipeline(file_path):
    df = load_csv(file_path)
    df["amount"] = df["amount"].astype(float)
    df.to_csv("processed_vendor_a.csv", index=False)

This made automation tasks predictable. If vendor B changes their file format, I know exactly which file to open and fix, no spelunking through unrelated logic.

Step 3: I Automated the Repetitive Setup Tasks

While I was okay with repeating processing logic in different vendor modules, I didn't want to keep rewriting boring setup code like file downloads and logging.

That's where selective abstraction came in:

import requests
import os

def download_file(url, save_path):
    r = requests.get(url)
    with open(save_path, 'wb') as f:
        f.write(r.content)

I reused this across vendors, but I didn't try to force everything into one shared function. Reuse was now intentional, not obsessive.

Step 4: I Added Clear Automation Entry Points

Instead of running one massive main.py with endless conditions, I gave each automation task its own CLI entry point.

Example using argparse:

import argparse
from vendors import vendor_a, vendor_b

parser = argparse.ArgumentParser()
parser.add_argument("vendor", choices=["a", "b"])
parser.add_argument("file_path")
args = parser.parse_args()

if args.vendor == "a":
    vendor_a.run_pipeline(args.file_path)
else:
    vendor_b.run_pipeline(args.file_path)

Now I could run:

python main.py a report.csv

and the right automation pipeline would run, without touching any unrelated code.

Step 5: Why Breaking DRY Made My Automation Faster to Maintain

Here's the thing: DRY is valuable when you're repeating identical logic that's unlikely to change. But in automation scripts, especially those dealing with multiple data sources, variation is the rule, not the exception.

By allowing small, purposeful repetition, I:

Reduced accidental breakage when editing code

Made each automation task readable in isolation

Spent less time debugging and more time adding features, as I like to put it: "Clarity beats cleverness in automation."

Pro Tip for Fellow Python Automators

If your automation code feels elegant but is a pain to modify, look for over-DRY patterns. Sometimes the cleanest code is the one that's okay with a little repetition as long as it's intentional.

If you like this post, then give 50 claps on this post and follow me.

Thanks for reading!

A message from our Founder

Hey, Sunil here. I wanted to take a moment to thank you for reading until the end and for being a part of this community.

Did you know that our team run these publications as a volunteer effort to over 3.5m monthly readers? We don't receive any funding, we do this to support the community. ❤️

If you want to show some love, please take a moment to follow me on LinkedIn, TikTok, Instagram. You can also subscribe to our weekly newsletter.

And before you go, don't forget to clap and follow the writer️!