Designing Python Scripts for Automation and Pipelines

A script is rarely the final destination. In pentest, forensics, and reverse engineering work, Python tools usually feed other tools: Bash pipelines, PowerShell pipelines, scanners, spreadsheets, or report generators. The way your script outputs data determines whether it can be reused.

The Core Idea

Make the output easy to consume and the interface flexible.

Bad output:

{'10.0.0.5', '172.16.0.10', '192.168.1.1'}

Good output:

10.0.0.5
172.16.0.10
192.168.1.1

The first one is a Python internal representation. The second is a format any other tool can parse.

Design Patterns

1. One Item Per Line

For lists of targets, IPs, domains, or hashes, print one per line. This is the universal format for tools like nmap, sort, uniq, wc, and xargs.

for target in sorted_targets:
    print(target)

2. Provide an Output File Option

When the user wants persistence, let them choose where to write:

parser.add_argument("-o", "--output", default="-",
                    help="output file (default: stdout)")

3. Default to stdout

If no output file is given, write to stdout. This lets your script plug into a pipeline:

python3 clean_targets.py targets_raw.txt | sort | uniq | nmap -iL -

Use sys.stdout as a default file object:

import argparse
import sys

parser = argparse.ArgumentParser()
parser.add_argument("input")
parser.add_argument("-o", "--output", default="-")
args = parser.parse_args()

if args.output == "-":
    outfile = sys.stdout
else:
    outfile = open(args.output, "w")

for target in targets:
    outfile.write(f"{target}\n")

4. Keep Status Messages on stderr

Progress messages, counts, and errors should go to sys.stderr so they do not pollute the data stream:

print(f"Wrote {len(targets)} targets", file=sys.stderr)

5. Use argparse for Real Tools

Hardcoded filenames like targets_clean.txt are fine for a classroom exercise, but real tools accept arguments:

import argparse

parser = argparse.ArgumentParser(description="Clean a target list")
parser.add_argument("input", help="raw target file")
parser.add_argument("-o", "--output", default="-", help="output file")
args = parser.parse_args()

Connection to PowerShell + Python Pipelines

This pattern is the same idea as the PowerShell + Python Acquisition Pipeline: one tool acquires or cleans, the next tool consumes. PowerShell and Bash both work best when each step produces clean, predictable text output.

Practical Test

Before submitting a script, ask:

1. Can another tool pipe this output directly?
2. Is the output format obvious and parseable?
3. Are errors separate from data?
4. Can the user choose the output location?

If the answer is yes, the script is automation-ready.