Database Migration, Part III
We're talking about extracting data from complex relational databases. This is -- in a way -- another case study for my Unlearning SQL book.
more ...We're talking about extracting data from complex relational databases. This is -- in a way -- another case study for my Unlearning SQL book.
more ...We're talking about extracting data from complex relational databases. This is -- in a way -- another case study for my Unlearning SQL book.
more ...Let's talk about extracting data from complex relational databases. This is -- in a way -- another case study for my Unlearning SQL book.
more ...In the process of writing Unlearning SQL, I had a need to extract SQL blocks from Python programs. Of course, I tried grep. It wasn't ideal.
Note
Book is available here:
Modern Python Cookbook — with lots and lots of recipes — is something you might need. Find the results of checking all these recipes here: https://www.amazon.com/Modern-Python-Cookbook-updated-techniques/dp/1835466389
I (reluctantly) switched from using mypy to using pyright to check all of these recipes carefully. The type alias (PEP …
more ...Book Announcement:
https://www.packtpub.com/en-us/product/modern-python-cookbook-9781835466384
There are about 130 recipes in here. Some new. Some revised. All examined (and tested) for Python 3.12.
For the Python folks who are just starting as well as those looking to pick up some more skills, this is for …
more ...See Synthetic Data.
I've updated the repository with a "Noisy Data" feature.
This will generate bulk data with invalid field values.
It helps with testing ETL pipelines to be sure they will scale to the expected volumes.
Clone https://github.com/slott56/DataSynthTool
Read https://slott56.github.io/DataSynthTool/_build …
more ...I've published a framework for doing SQL-like programming in Pure Python -- no database required.
Here: https://github.com/slott56/functional-SQL. See the functional-SQL documentation.
This allows us to transform SQL:
SELECT n.name, v.c2
FROM names_table n, values_table v
WHERE n.code = v.c1
To pure Python:
Select(name …
I've published a framework for making tools to leverage formal schema definitions to synthesize bulk data for performance tuning.
Here: https://github.com/slott56/DataSynthTool.
See the original talk.
See the documentation
more ...Book? Second draft (with tech review comments addressed) off to editors.
Boat? Still on the hard during Hurricane Season.
I've had a passing interest in data synthesis for decades.
Early on in my career, I figured out how the Z/OS IEDBG utility worked. See https://www.ibm …