See Synthetic Data.

I've updated the repository with a "Noisy Data" feature.

This will generate bulk data with invalid field values.

It helps with testing ETL pipelines to be sure they will scale to the expected volumes.

Clone https://github.com/slott56/DataSynthTool

Read https://slott56.github.io/DataSynthTool/_build …

I've published a framework for doing SQL-like programming in Pure Python -- no database required.

Here: https://github.com/slott56/functional-SQL. See the functional-SQL documentation.

This allows us to transform SQL:

SELECT n.name, v.c2
FROM names_table n, values_table v
WHERE n.code = v.c1

To pure Python:

Select(name …

I've published a framework for making tools to leverage formal schema definitions to synthesize bulk data for performance tuning.

Here: https://github.com/slott56/DataSynthTool.

See the original talk.

See the documentation

Book? Second draft (with tech review comments addressed) off to editors.

Boat? Still on the hard during Hurricane Season.

My sailing blog, Team Red Cruising is very large: 859 postings over the last few years. 2,334 image files.

This is a LOT of content.

A few of the files (were) Movies, which tend to create immense files.

The whole mess was so big

How Big Was It?

It …

Consider, for a moment, the shelve and dbm packages for storing things in a “database.” Built-in. Lightweight. The database is essentially a mapping from identifiers to objects. It can be quite nice.

The shelve module directly puts Python objects in a file. It’s an ideal database structure for Python …

I saw a confusing post on https://fosstodon.org.

I won't link to it, but I will quote it because it repeats some common misconceptions.

I have some iterator chain (in reality longer and more complex than this example).
And now in one or more steps, I need to add …

First, read this: Hazards And Safeguards for Software Rewrites. This is an infuriating article. The first part is nonsense. I wonder if it's intended to be read ironically? Maybe it's a list of bad ideas? Skip straight to the Safeguards for Rewrites section. It's really good.

Also, this Characterization Testing …

There are many good reasons to use shell scripts. Mostly, a script can be useful when it's an alias that launches an application. Beyond that, I have doubts.

I wasted three days because I refused to get a grip on how matplotlib really works.

Most of the time, folks like me are happy and successful using the pyplot module. The Quickstart provides brilliant, working examples.

As my partner's grandfather used to say, "We're off to the races in …