Complexity and Simplicity

Here's an interesting -- and common -- question.

"... any tools that I could use to create a web scraper that I could use to interact with a .aspx website?

I want to build a tool that will read an input file (e.g. an excel spreadsheet) containing a list of property parcel …

more ...

TDD and Python

First, let me say that TDD rocks.

Few things are as much fun as (1) writing a test script for a feature, and then (2) debugging the feature incrementally until it passes the test. It's fun because a great deal of hand-wringing and over-thinking is taken off the table.

To …

more ...

Python in the News

Date Tags #python

Making the rounds: Droopy: easy file receiving. Apparently, there were some widely-read blog posts about this. Google "Droopy: A Tiny Web Server That Makes Receiving Files a Snap" to see the buzz.

The point here is that 750 lines of Python code can go a long way. It's a complete …

more ...

A Limit to Reuse

We do a lot of bulk loads. A lot.

So many that we have some standard ETL-like modules for generic "Validate", "Load", "Load_Dimension", "Load_Fact" and those sorts of obvious patterns.

Mostly our business processes amount to a "dimensional conformance and fact load", followed by extracts, followed by a different "dimensional …

more ...


REST and HTTP Digest Authentication

It seems so simple: use the HTTP Digest Authorization with the Quality of Protection set to "auth".

It's an easy algorithm. A nonce that encodes a timestamp can be used to be sure no one is attempting to cache credentials. It's potentially very, very nice.

Except for one thing: Apache …

more ...

Getting Started Creating Web Pages

Got this question recently.

I’m looking for an HTML editor that fits into my price range (free of course). I don’t need to do anything fancy, just vanilla HTML to run on an Apache server ..., and maybe some PHP down the line. Can you recommend any open source …
more ...

COBOL File Processing in Python (really)

Years ago (6? 7?) I did some data profiling in Python.

This required reading COBOL files with Python code.

Superficially, this is not really very hard.

  1. Python slice syntax will pick fields on of the record. For example: data[12:14].

#. Python codecs will convert from EBCDIC to Unicode without …

more ...