I am a developer myself and would like to use latest technologies to build some open data datasets.

I would like to know if you are aware of any techniques or algorithm one can use to automate gather data on the internet instead of manually doing something following,

  • Google search and then trying to gather data
  • Put in Excel
  • Clean data using whatever language you require

I am fairly new to this so I might be thinking it completely wrong.

  • You should search for 'scraping websites'. There are plenty of tools but which one you can use best depends on the structure of the websites. (Not that I assume that data on the internet. Your question is unclear) – user4293 Jul 31 '17 at 13:25
  • Your 1st and 2nd paragraphs together suggest that you want to 'republish' connected data. Beware of copyright issues. – user4293 Jul 31 '17 at 13:27
  • 1
    'Not that I assume that data on the internet' -> 'Note that I assume that data on the internet are websites' – user4293 Jul 31 '17 at 13:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.