We do a lot of bulk loads. A lot.

So many that we have some standard ETL-like modules for generic "Validate", "Load", "Load_Dimension", "Load_Fact" and those sorts of obvious patterns.

Mostly our business processes amount to a "dimensional conformance and fact load", followed by extracts, followed by a different "dimensional …

Here's the situation.

A vendor sent us three separate files which need to be merged. 70,000+ records each. They're CSV files, so column position doesn't much matter. The column name (in row 1) is what matters.

I looked at three solutions. Two of which are merely OK. The third …

It seems so simple: use the HTTP Digest Authorization with the Quality of Protection set to "auth".

It's an easy algorithm. A nonce that encodes a timestamp can be used to be sure no one is attempting to cache credentials. It's potentially very, very nice.

Except for one thing: Apache …

Got this question recently.

I’m looking for an HTML editor that fits into my price range (free of course). I don’t need to do anything fancy, just vanilla HTML to run on an Apache server ..., and maybe some PHP down the line. Can you recommend any open source …

Years ago (6? 7?) I did some data profiling in Python.

This required reading COBOL files with Python code.

Superficially, this is not really very hard.

1. Python slice syntax will pick fields on of the record. For example: data[12:14].

#. Python codecs will convert from EBCDIC to Unicode without …

About a decade ago, I discovered the concept of Literate Programming. It's seductive. The idea is to write elegant documentation that embeds the actual working code.

For tricky, complex, high-visibility components, a literate programming approach can give people confidence that the software actually works as advertised.

I actually wrote my …

XKCD - http://xkcd.com/710/

http://en.wikipedia.org/wiki/Collatz_conjecture#Syracuse_function

I remember learning about this as an undergrad at Syracuse University in the 70's and didn't think much of it. It was just "one of those things" that I heard about, and perhaps wrote a homework assignment in …

Enterprise Applications really make people sweat. Look at this selection of StackOverflow questions. There are hundreds. People really get worked into a lather over this.

• What makes an application an “enterprise” or “enterprise-level” application?
• Is the LAMP stack appropriate for Enterprise use?
• What is “Enterprise ready”? Can we test for …

A year and change back, I wrote this on "Exploratory Programming".

It turns out that it was a mistake. While the subclass-expansion technique is a cool way to bang out a program incrementally, in the long run, the subclassing is ill-advised.

The more I look at Python generator functions and …

Interesting comments to Splitting Meta-Hairs.

Terms like Scripting, Interpreted and Dynamic are not "marketing terms". New, Improved, All Natural, Visual, Groovy, Lucid, etc., are marketing terms. I regret giving the impression that one should "not to spend time trying to get definitions that are exacting". One absolutely must get definitions …