I have a dataset of about half a million unique addresses. An address consists of a StreetAddress1, StreetAddress2, City, State and Zip code (all addresses are in the United States).
One of my main problems dealing with this dataset coherently is inconsistency in the wording of addresses. For example, "1 MAIN STREET" might also be phrased "ONE MAIN ST" referring to the same address.
Are there any free utilities that can help me remedy addresses in this very large set?
I was thinking about writing my own simple script to make addresses consistent with the following rules:
1. ROAD / STREET / AVENUE etc. are replaced by their abbrevations RD / ST / AVE 2. Written numbers ONE / TWO / THREE etc. are replaced by 1 / 2 / 3
However I'm concerned that there are a number of edge cases where this treatment will mangle valid addresses.
Are there existing utilities I can use to accomplish this, ideally locally on my own machine?