I looked through several related questions here (e.g. Tools for merging similar datasets continuously) but nothing seemed to quite answer my question. So, apologies in advance if I missed something.
I'm working on cleaning, normalizing, and linking some open data that consists of data dumps released as CSV files. New versions are released twice a month.
I want to include the new versions (which have additional data as well as possibly corrections of old data) without "losing my place" as it were with the normalizing I've already done. What's the best way to manage regular, periodic updates of CSV data?
Out of the box "diff" tools don't offer much for CSV. I don't mind doing some scripting (python) to solve the problem but I'm not sure of the best approach for tracking, reviewing, and incorporating updates to data received in this form.