I got a long list of requirements from a firm that's looking to build a related family of web sites. They were down to a Django vs. Ruby-on-Rails decision.

As you can see, they've done their homework in thinking through their needs.

I grouped their "high-level requirements" into several categories …

The multiprocessing module includes a generic Process class, which can be used to wrap a simple function.

The function must be designed to work with Queues or Pipelines or other synchronization techniques.

There's an advantage, however, to defining a class which gracefully handles generator functions. If we have Generator-Aware multi-processing …

PDF files aren't pleasant.

The good news is that they're documented (http://www.adobe.com/devnet/pdf/pdf_reference.html).

They bad news is that they're rather complex.

I found four Python packages for reading PDF files.

• http://pybrary.net/pyPdf/ - weak
• http://www.swftools.org/gfx_tutorial.html - depends on binary …

The advantage of multiprocessing is to have multiple processes working on a problem. If we break a big problem into small, concurrent steps, we can often get results in less elapsed time by making more effective use of the CPU. Specifically, we want to make use of non-user time where …

See Apache Log Parsing for the background.

Here's a generator function which expands a simple Access to be a more detailed named tuple.

Access_Details= namedtuple( 'Access_Details',
    ['access', 'time', 'method', 'url', 'protocol'] )

def details_iter( iterable ):
    def parse_time_linux( ts ):
        return datetime.datetime.strptime( ts, "%d/%b/%Y:%H:%M:%S %z" )

    def …

How much do I love Python? Consider this little snippet that parses Apache logs.

import re
from collections import defaultdict, named tuple

format_pat= re.compile(
   r"(?P<host>[\d\.]+)\s"
   r"(?P<identity>\S*)\s"
   r"(?P<user>\S*)\s"
   r"\[(?P<time>.*?)\]\s"
   r'"(?P<request>.*?)"\s'
   r"(?P …

I've revised (and streamlined) my Building Skills in Programming book. The 2.6.2. edition will simply replace the 2.6.1. edition, leading to the possibility of broken bookmarks because of the changes.

Currently, the non-programmer book accounts for under 10% hits on the http://www.itmaybeaback.com/book …

The csv module in Python 2.7 is more-or-less hard-wired to work with ASCII and only ASCII.

Sadly, we're often confronted with CSV files that include Unicode characters. There are numerous Stack Overflow questions on this topic. http://stackoverflow.com/search?q=python+csv+unicode

What to do? Since csv …

Read this: Shared Counter with Python Multiprocessing.

Brilliant. Thank you for this.

Too many of the questions on StackOverflow that include multi-threading are better approached as multi-processing. In Linux, there are times when all threads of a single process are stopped while the process (as a whole) waits for system …