Another case study on TDRE.

Provided: 2,938 lines of Python code which process a handful of large files to create a number of outputs. [Details can't be disclosed.]

Objective: Refactor to distinguish between the overall sequence of transformational steps and the details of each individual step.

Half the fun on Stack Overflow is the endless use of closed-ended questions. "Can I do this in Python?" being so common and so hilarious.

The answer is "Yes." You can do it.

Perhaps that's not the question they really meant to ask.

See "Open versus Closed Ended Questions" for …

I've done a fair amount of reverse engineering over the years.

In the early days, you went from code to specification to new code. It took forever and the problems you uncovered -- well -- they often derailed the project.

Recently, I used a TDD-like approach. Each piece of legacy code was …

Python is a top-shelf toolset for creating sample data to do performance testing.

Let's say that you need to validate a data warehouse design, and you need a million facts that join with thousands of dimension entities across a half-dozen dimensions. You'll be generating data for seven different tables, and …

At this point, we have clex.py, which uses sqlpreproc.py to create a proper lexer for C source code. We use PLY 's ANSI C parser (cparse.py) as the backbone of our own analysis of C.

The C Language.

Separate from the lexical structure of C (the spelling …

The client said, "We have this program, largely in C, which we can no longer support. It doesn't really meet our business needs and it doesn't fit our technology skill set." As part of rewriting the requirements, the question of what it really did came up.

Actually, they had more …

A stored procedure isn't really very easy to understand. There's a profound fascination with triggers and stored procedures, and they're both really bad ideas. I can't say enough bad things about stored procedures. See PL/SQL vs. Java - Which is REALLY faster? and Over-Solving the Problem or When your architect …

Python is Batteries Included ™ programming. These analysis tools are either Python Out Of the Box , or they are straight-forward downloads of other open-source components.

Here are some analytical situations where Python has saved my bacon. I'll present some code for several of these.

1. Legacy Data Domains. There are legacy (in …