I am a developer myself and would like to use latest technologies to build some open data datasets.
I would like to know if you are aware of any techniques or algorithm one can use to automate gather data on the internet instead of manually doing something following,
- Google search and then trying to gather data
- Put in Excel
- Clean data using whatever language you require
I am fairly new to this so I might be thinking it completely wrong.